Re: [jira] [Work logged] (MAHOUT-2147) Add scaladocs for Canopy Clutering Algorithm

[Andrew Palumbo]

I don't think We ever ported canopy clustering over from hadoop.. +1 on closing 
this and any similar..

Sent from my Verizon, Samsung Galaxy smartphone
Get Outlook for Android

From: ASF GitHub Bot (Jira) 
Sent: Friday, February 3, 2023 10:42:00 AM
To: iss...@mahout.apache.org 
Subject: [jira] [Work logged] (MAHOUT-2147) Add scaladocs for Canopy Clutering 
Algorithm


 [ 
https://issues.apache.org/jira/browse/MAHOUT-2147?focusedWorklogId=843550=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-843550
 ]

ASF GitHub Bot logged work on MAHOUT-2147:
--

Author: ASF GitHub Bot
Created on: 03/Feb/23 18:41
Start Date: 03/Feb/23 18:41
Worklog Time Spent: 10m
  Work Description: rawkintrevo opened a new pull request, #421:
URL: https://github.com/apache/mahout/pull/421

   ### Purpose of PR:

   Add scaladocs for Canopy Cluster Algorithm


   ### Important ToDos
   Please mark each with an "x"
   - [x] A JIRA ticket exists (if not, please create this 
first)[https://issues.apache.org/jira/browse/mahout/]
   - [x] Title of PR is "MAHOUT- Brief Description of Changes" where  
is the JIRA number.
   - [x] Assigned JIRA to self
   - [x] Added documentation in scala docs/java docs, and to website

   Does this change break earlier versions? No

   Is this the beginning of a larger project for which a feature branch should 
be made? No





Issue Time Tracking
---

Worklog Id: (was: 843550)
Remaining Estimate: 0h
Time Spent: 10m

> Add scaladocs for Canopy Clutering Algorithm
> 
>
> Key: MAHOUT-2147
> URL: https://issues.apache.org/jira/browse/MAHOUT-2147
> Project: Mahout
>  Issue Type: Improvement
>Reporter: Trevor Grant
>Assignee: Trevor Grant
>Priority: Trivial
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [jira] [Created] (MAHOUT-2136) Implement Ridge Regression Algorithm

[Andrew Palumbo]

+1

From: Jose Francisco Hernandez Santa Cruz (Jira) 
Sent: Friday, January 8, 2021 12:03 PM
To: dev@mahout.apache.org 
Subject: [jira] [Created] (MAHOUT-2136) Implement Ridge Regression Algorithm

Jose Francisco Hernandez Santa Cruz created MAHOUT-2136:
---

 Summary: Implement Ridge Regression Algorithm
 Key: MAHOUT-2136
 URL: https://issues.apache.org/jira/browse/MAHOUT-2136
 Project: Mahout
  Issue Type: Improvement
Reporter: Jose Francisco Hernandez Santa Cruz


Ridge regression is a regularized approach of the linear regression by OLS 
method. Non-full rank matrices are allowed as the regularization parameter adds 
a diagonal matrix to the optimization equation breaking collinearity by adding 
a bias component, thus reducing standard error.



https://en.wikipedia.org/wiki/Tikhonov_regularization



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Resuming Community Calls

[Andrew Palumbo]

I couldn't make last week.  see you all  tomorrow.

From: Trevor Grant 
Sent: Tuesday, January 5, 2021 3:43 PM
To: Mahout Dev List 
Subject: Re: Resuming Community Calls

Apologies- I forgot to add the meeting link:
https://hangouts.google.com/hangouts/_/calendar/YW5kcmV3Lm11c3NlbG1hbkBnbWFpbC5jb20.4tg9vg3t51gjlco0ebl5siim1s


On Tue, Jan 5, 2021 at 7:27 AM Trevor Grant 
wrote:

> Hey Mahout Dev,
>
> We're going to resume our community calls on Tuesdays tonight (or this
> afternoon or tomorrow morning, depending where you are in the world).
>
> 22:00 UTC
> 20:00 Greenwich Mean Time
> 5:00 PM Eastern Standard
> 4:00 PM Central Standard
> 2:00 PM Pacific Standard
>
> Plan to discuss on going projects people have been working on and upcoming
> board report.
>
> Will reflect minutes to website and other important updates to mailing
> list. Hope y'all are having a good year, and look forward to seeing you
> there.
>
> Trevor
>

Re: PyMahout (incore) (alpha v0.1)

[Andrew Palumbo]

+1

From: Andrew Musselman 
Sent: Thursday, January 7, 2021 2:46 PM
To: u...@mahout.apache.org 
Cc: Mahout Dev List 
Subject: Re: PyMahout (incore) (alpha v0.1)

Thanks Trevor, looking forward to trying it out.

On Wed, Jan 6, 2021 at 5:30 PM Peng Zhang  wrote:

> Well done Trevor.
>
> -peng
>
> On Thu, Jan 7, 2021 at 04:45 Trevor Grant 
> wrote:
>
> > Hey all,
> >
> > I made a branch for a thing I'm toying with. PyMahout.
> >
> > See https://github.com/rawkintrevo/pymahout/tree/trunk
> >
> > Right now, its sort of dumb- it just makes a couple of random incore
> > matrices... but it _does_ make them.
> >
> > Next I want to show I can do something with DRMs.
> >
> > Once I know its all possible- Ill make a batch of JIRA tickets and we can
> > start implementing a python like package so that in theory in a pyspark
> > workbook you could
> >
> > ```jupyter
> > !pip install pymahout
> > 
> >
> > import pymhout
> >
> > # do pymahot things here... in python.
> >
> > ```
> >
> > So if you're interested in helping /playing- reach out on here or direct-
> > if there is a bunch of interest I can commit all of this to a branch as
> we
> > play with it.
> >
> > Thanks!
> > tg
> >
>

[Discuss] Vague Thoughts for a Possible direction for the ViennaCL module (quick sketch)

[Andrew Palumbo]

Hey all,
I started a half thought on slack the other day, got pulled away before I could 
finish.  This is just a quick writeup, which needs more citations and empirical 
evidence to back up the reasoning, along with some examples.

I was looking at some FPGA work, and had a thought.  We have two ViennaCL 
modules in the experimental section of the project, based on the ViennaCL 
Abstract BLAS library[1][2]: mahout-viennacl-omp_2.x and mahout-viennacl_2.x 
which parallelize routines on shared memory CPU cores via OpenMP [2], and 
offload routines to GPU using OpenCL[4] respectively.

We've had good luck running the OpenMP module on off- heap main memory, showing 
significant speedups in Level II and level III BLAS operations, both sparse and 
dense matrix algebra, which provide speedups on algorithms like DSSVD and DSPCA 
[?], which are iterative and heavily dependent on Matrix-Matrix multiplication. 
 This module, with some clean-up of the module, itself [5], and the 
auto-probing logic [6].  While there is still some work to be done on memory 
management in this module, it is trivial, i.e. ensure minimal memory copies are 
made during an evaluation trigger [7].  This module could be tuned, cleaned and 
me ready for testing, to allow it back in the production environment.

However the the ViennaCL OpenCL module is still very much in an expiremental 
phase.  While significant speedups have been shown in both sparse and dense 
matrix algebra [8]; up to 30x on trivial matrix matrix algebra problems, where 
matrix blocks per node fit into a node's GPU memory, no memory tiling or 
scatter/gather routines are implemented for Multiple GPUs on a node [9] at the 
time of this writhing, and memory management is naive, with a copy from of-heap 
to GPU memory at each operation; a memory management scheme using 
synchronization to main memory operands and solutions has been proposed [9], 
but not implemented therefore the iterative evaluation of e.g.

 drmA := ...
 drmB  := ...
  for i <- 1..10 do
 drmC := mxRAND(b.ncol,100)
 drmA <- drmB * drmC
  done

would incur 10 copies per block of both mxA and  mxB back end blocks to and 
from the GPU.

Considering that a tiling strategy is not yet implemented for multiple GPUs, 
there is signigicant work to do to get this mpodule up the state of the art for 
a Spark backend.

Furthermore, work on CUDA backend has already begun, with a branch, 
apache/mahout:[CUDA], which makes use of the JCuda library, directly 
interfacing with the CuBLAS, CuSparse and other CUDA Libraries, which have had 
much work in memory management, shared memory, and optimized routines in the 
years since the ViennaCL library were written.

It was decided in [xx] that further GPU work (For NVIDIA libraries), and 
ViennaCL has been moved into the expiremental sub-project.

This leaves us with a ViennaCL library, partly finished, which could be useful 
still for AMD GPUs, however OpenCL itself being an abstraction layer leaves 
open the opportunities to begin experimental work on some other hardware. e.g 
FPGA[9][10]s.

Prpoposal:
Finish out basis GPU Blas routines in the apache/mahout:[CUDA] branch, and 
repurpose the ViennaCL for FPGAs pipelines on FPGA[10][11]. fBLAS demonstrates 
the offloading matrix math OpenCL Kernels to FPGA Matrix math backends [12].   
While  fBLAS is an example the Khronos group has enlisted a large group of 
hardware and software venodoors to adopth the OpenCL standard[13].

For an example of Khornos Groups Vector Add HPP OpenCL kernel see [14].

Much investigation would need to be done into optimal methods of FPGA 
prgrogramming.  E.g would it be possible to "Jit" pipelines as set forth in 
fBLAS (generate OpenCL kernels from tree at evaluation time after optimization 
to native code and compile then stream data through?)

Or in the case of e.g  a NN, optimize iteratively, and deliver a finished, set 
of generated code, or even set of programmed FPGAs derived from a mahout 
algorithm, optimized on a distributed dataset?

These are just thoughts of a way to move forward with an OpenCL Module, which 
we have in the experimental sub project currently.

Work has been put forward in the Apache environment [15], and translations from 
tensorflow to ViennaCL with the inten of extending to a generalized FPGA 
framework can be seen here[16] .

Intel's OneAPI[20]. while a reach is something that could be explored.

AWS FPGA instances[21]


[1] http://viennacl.sourceforge.net/
[2]http://viennacl.sourceforge.net/doc/manual-installation.html#manual-installation-backends
[3] https://www.openmp.org//
[4]https://www.khronos.org/opencl/
[5] MAHOUT-
[6] MAHOUT-
[7]https://www.ibm.com/support/knowledgecenter/SSYKE2_8.0.0/com.ibm.java.vm.80.doc/docs/jni_sync.html
[8] Mahout ViennaCL Bindings#Memory Syncronizatrion
[xx]
[9]https://www.khronos.org/opencl/resources

Re: [DISCUSS] Move time of weekly meeting

[Andrew Palumbo]

I may be able to come to some Tuesday Afternoon meetings.  CST or PST?



From: Andrew Musselman 
Sent: Monday, November 23, 2020 5:43 PM
To: Mahout Dev List 
Subject: Re: [DISCUSS] Move time of weekly meeting

Hi Trevor, thanks for asking; I've actually gotten to where Friday mornings
are a de facto catch-up at work time.. I might be better off on Monday or
Tuesday mid-afternoons.

On Fri, Nov 20, 2020 at 10:23 AM Trevor Grant 
wrote:

> Hey all, slow meeting this week[1]- was wondering if anyone would like to
> move the meeting and if so  to when?
>
> [1]
> http://mahout.apache.org/minutes/2020/11/20/weekly-meeting-minutes.html
>

Re: [ANNOUNCE] New Committer: Christofer Dutz

[Andrew Palumbo]

welcome aboard, Chis! thank you for all the work on the build!

--andy

From: Andrew Musselman 
Sent: Monday, June 8, 2020 7:23 PM
To: Christofer Dutz 
Cc: Mahout Dev List 
Subject: Re: [ANNOUNCE] New Committer: Christofer Dutz

You bet, welcome aboard :)

Chris, I've set your commit bit on the project; you should be able to
commit to ASF gitbox or to GitHub.

On Mon, Jun 8, 2020 at 2:01 PM Christofer Dutz 
wrote:

> Hi folks
>
> And thanks for inviting me :-)
>
> Happy to continue to be of assistance.
>
> Chris
> --
> *Von:* Andrew Musselman 
> *Gesendet:* Montag, 8. Juni 2020 19:10
> *An:* Mahout Dev List 
> *Betreff:* Re: [ANNOUNCE] New Committer: Christofer Dutz
>
> Well deserved, a great addition to the team! Welcome Chris!
>
> On Mon, Jun 8, 2020 at 8:39 AM Trevor Grant 
> wrote:
>
> > The Project Management Committee (PMC) for Apache Mahout
> > has invited Christofer Dutz to become a committer and we are pleased
> > to announce that they have accepted.
> >
> > Chris has been a huge help in assisting us with cleaning up the Mahout
> > build.
> >
> > Being a committer enables easier contribution to the
> > project since there is no need to go via the patch
> > submission process. This should enable better productivity.
> >
>

Re: Re: Hi ... need some help?

[Andrew Palumbo]

Hey All, been very sick again,  I may be offline for a couple of weeks,I am 
probably not going to be able to commit to times in the next couple of weeks,  
I'll be back and forth to SD and sleeping alot.  Thank you again, Christofer, 
for the help.  If anyone needs me for anything, please use my 
apalu...@apache.org address, (i am not subscribed to mahout with that) or give 
a shout on slack, so that i dont miss it.

Thanks.

Andy

-- Forwarded message --
From: Christofer Dutz 
Date: Apr 24, 2020 2:18 AM
Subject: Re: Hi ... need some help?
To: dev@mahout.apache.org
Cc:


Hi all,

just found the email with the time for the call today ... that's 19:00 / 7pm 
here ... perfect.
Would just need some information on how to join.

Chris


Am 20.04.20, 17:31 schrieb "Andrew Musselman" :

Thanks Chris, will take a look at your PR.

I think we would be fine upgrading anything that is still making
improvements, probably makes sense to discuss Friday on our call if you can
make it.

Best
Andrew

On Mon, Apr 20, 2020 at 1:53 AM Christofer Dutz 
wrote:

> Hi Folks,
>
> so I've now tested the build with java 1.8, 9, 10 and they work ...
> 11 I'm getting errors about unsupported java major versions again ...
> guess there's some old library version in there somewhere.
>
> But at least I managed to get you out of this 1.7 trap.
>
> Chris
>
>
> Am 20.04.20, 09:46 schrieb "Christofer Dutz" :
>
> Hi folks,
>
> so I was now able to build (including all tests) with Java 8 and 9 ...
> currently trying 10 ...
>
> Are there any objection that some maven dependencies get updated to
> more recent versions? I mean ... the hbase-client you're using is more 
than
> 5 years old ...
>
> Chris
>
>
> Am 20.04.20, 00:29 schrieb "Andrew Musselman" <
> andrew.mussel...@gmail.com>:
>
> No problem; would 10:00 a.m. Pacific next Friday the 24th work for
> you time
> zone-wise?
>
> On Sun, Apr 19, 2020 at 12:15 PM Christofer Dutz <
> christofer.d...@c-ware.de>
> wrote:
>
> > Sorry ...
> >
> > didn't see your response ... into Mahout too deep ;-)
> > Guess we have to postpone this to sometime over the week.
> >
> > Chris
> >
> >
> > Am 19.04.20, 19:56 schrieb "Andrew Musselman" <
> andrew.mussel...@gmail.com
> > >:
> >
> > *what time
> >
> > On Sun, Apr 19, 2020 at 10:51 Andrew Musselman <
> > andrew.mussel...@gmail.com>
> > wrote:
> >
> > > I think it's safe to move to 1.8; yeah what tune is good
> for you?
> > I'm in
> > > Pacific time zone and am flexible this afternoon.
> > >
> > > Trevor you free?
> > >
> > > On Sun, Apr 19, 2020 at 09:37 Christofer Dutz <
> > christofer.d...@c-ware.de>
> > > wrote:
> > >
> > >> Yikes! ... well guess then I can't help you folks as it's
> almost
> > >> impossible to get my hands on a 1.7 version.
> > >>
> > >> What's preventing you from going to 1.8+?
> > >>
> > >> Any your build says 1.8 and above:
> > >> 
> > >>   [1.8,)
> > >> 
> > >>
> > >> Regarding the artifacts ... would it be ok to have the
> maven
> > artifacts
> > >> using classifiers and perhaps the files in the libs and
> > distribution to
> > >> follow the typical scala sheme?
> > >>
> > >> Chris
> > >>
> > >> Am 19.04.20, 17:56 schrieb "Trevor Grant" <
> trevor.d.gr...@gmail.com
> > >:
> > >>
> > >> Yea, we have a requirement on 1.7. we need to get it
> up to 8,
> > but
> > >> considered that a different issue.
> > >>
> > >> Maven throws warning, sbt breaks down entirely (when
> importing)-
> > >> hence why
> > >> we were using a script to replace 2.11 w 2.12
> > >>
> > >> On Sun, Apr 19, 2020, 10:33 AM Christofer Dutz <
> > >> christofer.d...@c-ware.de>
> > >> wrote:
> > >>
> > >> > Well I have been compiling with various jdks from 8
> to 14.
> > However I
> > >> > noticed that if I select a 

Re: Hi ... need some help?

[Andrew Palumbo]

n general.


Thank you, makes sense.

Today I'll go through the poms again managing all versions and cleaning up the 
order of things. Then if all still works I would bump the dependencies versions 
up as much as possible.

Will report back as soon as I'm though or I've got something to report ... then 
I'll also go into details with your feedback (I haven't ignored it ;-) )


Thanks again!




Chris

[1]https://github.com/apache/mahout/tree/master/core/src/main
[2]https://github.com/apache/mahout/blob/master/community/community-engines/h2o/pom.xml#L91



Am 22.04.20, 06:08 schrieb "Andrew Palumbo" :

Fixing previous message..


Quote from Chris Dutz:

> Hi folks,
>so I was now able to build (including all tests) with Java 8 and 9 ... 
currently trying 10 ...
>Are there any objection that some maven dependencies get updated to 
more recent versions? I mean ... the hbase-client you're using is more than 5 
years old ...

My answer:

I personally have no problem with the updating of any dependencies, they 
may break some things and caue more work, but that is the kind of thing that 
we've been trying to get done in this build work,  get everything up to speed.

Id say take Andrew, Trevor and Pat's word over mine though i am a bit less 
active presently.

Thanks.

Andy

________
From: Andrew Palumbo 
Sent: Tuesday, April 21, 2020 10:17 PM
To: dev@mahout.apache.org 
Subject: Re: Hi ... need some help?

  Hi folks,

so I was now able to build (including all tests) with Java 8 and 9 ... 
currently trying 10 ...

Are there any objection that some maven dependencies get updated to 
more recent versions? I mean ... the hbase-client you're using is more than 5 
years old ...
Not by me, I believe that is being used by the MR module, which is 
Deprecated.

I personally have no problem with the updating of any dependencies, they 
may break some things and caue more work, but that is the kind of thing that 
we've been trying to get done in this build work,  get everything up to speed.

Id say take Andrew, Trevor and Pat's word over mine though i am a bit less 
active presently.

Thanks.

Andy
________
From: Andrew Palumbo 
Sent: Tuesday, April 21, 2020 10:13 PM
To: dev@mahout.apache.org 
Subject: Re: Hi ... need some help?

Chris, Thank you so much for what you are doing,  This is Apache at its 
best.. I've been down and out with a serious Illness, Injury and other issues, 
which have seriously limited my Machine time.   I was pretty close to getting a 
good build, but it was hacky, and the method that you use to name the modules 
for both Scala versions, looks great.

We've always relied on Stevo to fix the builds for us,  but as he said is 
unable to contribute right now.  The main issues (solved by hacks), currently 
are


  1.  Dependencies and transitive dependencies  are not being picked and 
copied to the `./lib` directory, where `/bin/mahout` and parts of the 
MahoutSparkContext look for them, to add to the class path.  So running either 
from the CLI or as a library, dependencies are not picked up.
 *   We used to use the mahout-experimental-xx.jar as a fat jar for 
this, though it was bloated with now deprecated MR stuff, and no longer packed.
  2.  `./bin/mahout` (and `compute-classpath.sh`) need to be revamped to 
ensure that they are picking up the correct classes.

w.r.t. to Java 8/7 issues, We did mandate Java 8+, and this required a few 
minor code changes to play nicely with Scala 2.11.  Mainly one class needed a 
JVM "Static" field, so i refactored that field out of the Class and into a 
companion object.  I wonder if this is what is giving you issues with Java 7.

I'd thought that Java 8 was mandated now, but may be thinking of maven 
3.3.x.

Regardless Thank you very much for this.  This board is doing really doing 
well so far. and deserves accolades.

>
> 
> org.apache.mahout
> mahout-spark
> 14.1-SNAPSHOT
> 2.11
> 
This would be perfect IMO.


I can send you the commits that I am talking about.

As well, I saw that Trevor gave you a link to a filter.. I have one here 
with a bit more limited scope, which is open issues fixversion == 14.1.

To answer one question yes this was recently building, and releasing, with 
all of the tests passing (for a few modules, that we were focusing on).  after 
that i made some changes that broke it again..

the board with limited scope: 
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=348=detail=MAHOUT-2093


Thanks again for helping out.  we are really bad with poms, not so much 
from the ground up, as fixing some that are 10 years old, as Stevo mentioned, 
very quickly while working on several othe

Re: Hi ... need some help?

[Andrew Palumbo]

From: Christofer Dutz 
Sent: Wednesday, April 22, 2020 4:46 AM
To: dev@mahout.apache.org 
Subject: Re: Hi ... need some help?

Hi Andrew,

thanks for your kind words ... they are sort of the fuel that makes me run ;-)

So some general observations and suggestions:
- You seem to use test-jars quite a bit: These are generally considered an 
antipattern as you possibly import problems from another module and you will 
have no way of detecting them. If you need shared test-code it's better 
practice to create a dedicated test-utils module and include that wherever it's 
needed.

Aha yeah I see that Flink has one.  Yes agreed, in general,   though here,  the 
tests which have setup() and teardown() type routines, are Engine specific.  We 
tried to keep everything Engine neutral from the top down to the tests which 
were specific to H2O, Spark and Flink's Implementation of the abstract top 
level DSL (core (java/o.a.m.math)and more speciofically scala/o.a.mmath-scala). 
  I'd like to say that we designed it like that, maybe we did.. or maybe it was 
a copy/paste [1]



The H2O module is huge, because H20 is written in Java, and doesn't provide 
Scala libraries for scala compile, etc. [2].


- Don't use variables for project dependencies: It makes things slightly more 
difficult to read the release plugin takes care of updating version for you and 
some third party plugins might have issues with it.


   I see that you fixed it.. Great thx.  Yeah I was pulling my hair out 
after Andrew and Trevor did at that point.. I didn't figure it was best 
practices, but i looked around and saw a lot scripts with sed -i /_2.11/2.12/g, 
e.g. I see that you changed most everything.  Just briefly looked over root 
pom.xml, glad you took this over, we would have had another mess..

- I usually provide versions for all project dependencies and have all other 
dependencies managed in a dependencyManagement section of the root module this 
avoids problems with version conflicts when constructing something using 
multiple parts of your projects (Especially your lib directory thing)

   Thx yes makes sense again.   I wasn't aware of the method that you used 
to handle that transitive deps in the root pom.xml


- Accessing resources outside of the current modules scope is generally 
considered an antipattern ... regarding your lib thing, I would suggest an 
assembly that builds a directory (but I do understand that this version perhaps 
speeds up the development workflow ... we could move the clean plugin 
configuration and the antrun plugin config into a profile dedicated for 
development)

Yes this last refactor had moved several things around, I have ideas to 
change a bit more that I need to bring up sometime..  there were a few reasons 
for this..




- I usually order the plugin configurations (as much as possible) the way they 
are usually executed in the build ... so: clean, process resources, compile, 
test, package, ... this makes it easier to understand the build in general.

Today I'll go through the poms again managing all versions and cleaning up the 
order of things. Then if all still works I would bump the dependencies versions 
up as much as possible.

Will report back as soon as I'm though or I've got something to report ... then 
I'll also go into details with your feedback (I haven't ignored it ;-) )

Chris

[1]https://github.com/apache/mahout/tree/master/core/src/main
[2]https://github.com/apache/mahout/blob/master/community/community-engines/h2o/pom.xml#L91



Am 22.04.20, 06:08 schrieb "Andrew Palumbo" :

Fixing previous message..


Quote from Chris Dutz:

> Hi folks,
>so I was now able to build (including all tests) with Java 8 and 9 ... 
currently trying 10 ...
>Are there any objection that some maven dependencies get updated to 
more recent versions? I mean ... the hbase-client you're using is more than 5 
years old ...

My answer:

I personally have no problem with the updating of any dependencies, they 
may break some things and caue more work, but that is the kind of thing that 
we've been trying to get done in this build work,  get everything up to speed.

Id say take Andrew, Trevor and Pat's word over mine though i am a bit less 
active presently.

Thanks.

Andy

____
From: Andrew Palumbo 
Sent: Tuesday, April 21, 2020 10:17 PM
To: dev@mahout.apache.org 
Subject: Re: Hi ... need some help?

  Hi folks,

so I was now able to build (including all tests) with Java 8 and 9 ... 
currently trying 10 ...

Are there any objection that some maven dependencies get updated to 
more recent versions? I mean ... the hbase-client you're using is more than 5 
years old ...
Not by me, I believe that is being used by the MR module, which is 
Deprecated.

I personally have no problem with the updating of any dependencies, t

Re: Hi ... need some help?

[Andrew Palumbo]

Fixing previous message..


Quote from Chris Dutz:

> Hi folks,
>so I was now able to build (including all tests) with Java 8 and 9 ... 
> currently trying 10 ...
>Are there any objection that some maven dependencies get updated to more 
> recent versions? I mean ... the hbase-client you're using is more than 5 
> years old ...

My answer:

I personally have no problem with the updating of any dependencies, they may 
break some things and caue more work, but that is the kind of thing that we've 
been trying to get done in this build work,  get everything up to speed.

Id say take Andrew, Trevor and Pat's word over mine though i am a bit less 
active presently.

Thanks.

Andy

____
From: Andrew Palumbo 
Sent: Tuesday, April 21, 2020 10:17 PM
To: dev@mahout.apache.org 
Subject: Re: Hi ... need some help?

  Hi folks,

so I was now able to build (including all tests) with Java 8 and 9 ... 
currently trying 10 ...

Are there any objection that some maven dependencies get updated to more 
recent versions? I mean ... the hbase-client you're using is more than 5 years 
old ...
Not by me, I believe that is being used by the MR module, which is Deprecated.

I personally have no problem with the updating of any dependencies, they may 
break some things and caue more work, but that is the kind of thing that we've 
been trying to get done in this build work,  get everything up to speed.

Id say take Andrew, Trevor and Pat's word over mine though i am a bit less 
active presently.

Thanks.

Andy
____
From: Andrew Palumbo 
Sent: Tuesday, April 21, 2020 10:13 PM
To: dev@mahout.apache.org 
Subject: Re: Hi ... need some help?

Chris, Thank you so much for what you are doing,  This is Apache at its best.. 
I've been down and out with a serious Illness, Injury and other issues, which 
have seriously limited my Machine time.   I was pretty close to getting a good 
build, but it was hacky, and the method that you use to name the modules for 
both Scala versions, looks great.

We've always relied on Stevo to fix the builds for us,  but as he said is 
unable to contribute right now.  The main issues (solved by hacks), currently 
are


  1.  Dependencies and transitive dependencies  are not being picked and copied 
to the `./lib` directory, where `/bin/mahout` and parts of the 
MahoutSparkContext look for them, to add to the class path.  So running either 
from the CLI or as a library, dependencies are not picked up.
 *   We used to use the mahout-experimental-xx.jar as a fat jar for this, 
though it was bloated with now deprecated MR stuff, and no longer packed.
  2.  `./bin/mahout` (and `compute-classpath.sh`) need to be revamped to ensure 
that they are picking up the correct classes.

w.r.t. to Java 8/7 issues, We did mandate Java 8+, and this required a few 
minor code changes to play nicely with Scala 2.11.  Mainly one class needed a 
JVM "Static" field, so i refactored that field out of the Class and into a 
companion object.  I wonder if this is what is giving you issues with Java 7.

I'd thought that Java 8 was mandated now, but may be thinking of maven 3.3.x.

Regardless Thank you very much for this.  This board is doing really doing well 
so far. and deserves accolades.

>
> 
> org.apache.mahout
> mahout-spark
> 14.1-SNAPSHOT
> 2.11
> 
This would be perfect IMO.


I can send you the commits that I am talking about.

As well, I saw that Trevor gave you a link to a filter.. I have one here with a 
bit more limited scope, which is open issues fixversion == 14.1.

To answer one question yes this was recently building, and releasing, with all 
of the tests passing (for a few modules, that we were focusing on).  after that 
i made some changes that broke it again..

the board with limited scope: 
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=348=detail=MAHOUT-2093


Thanks again for helping out.  we are really bad with poms, not so much from 
the ground up, as fixing some that are 10 years old, as Stevo mentioned, very 
quickly while working on several other things.

Thank you again for this. It is a great help, and once we get a good build, we 
can get back to doing work on the library itself.

I have some documents that i can provide if it will help explain the structure 
of the project, which is still kind of in flux.  E.g. I'd like to get the 
ViennaCL-OMP branch out of experimental, but there is much to clean up first.  
As well, I am on medical leave, and dont have much time on the computer these 
days.. have to budget my time.

I'll send you some (closed) PRs with notes and changes, if it helps.  lmk.

Thanks again, This is Huge.

Andy

From: Christofer Dutz 
Sent: Thursday, April 16, 2020 9:50 AM
To: dev@mahout.apache.org 
Subject: Re: Hi ... need some help?

Hi Trevor,

ok ... first of all ... t

Re: Hi ... need some help?

[Andrew Palumbo]

  Hi folks,

so I was now able to build (including all tests) with Java 8 and 9 ... 
currently trying 10 ...

Are there any objection that some maven dependencies get updated to more 
recent versions? I mean ... the hbase-client you're using is more than 5 years 
old ...
Not by me, I believe that is being used by the MR module, which is Deprecated.

I personally have no problem with the updating of any dependencies, they may 
break some things and caue more work, but that is the kind of thing that we've 
been trying to get done in this build work,  get everything up to speed.

Id say take Andrew, Trevor and Pat's word over mine though i am a bit less 
active presently.

Thanks.

Andy

From: Andrew Palumbo 
Sent: Tuesday, April 21, 2020 10:13 PM
To: dev@mahout.apache.org 
Subject: Re: Hi ... need some help?

Chris, Thank you so much for what you are doing,  This is Apache at its best.. 
I've been down and out with a serious Illness, Injury and other issues, which 
have seriously limited my Machine time.   I was pretty close to getting a good 
build, but it was hacky, and the method that you use to name the modules for 
both Scala versions, looks great.

We've always relied on Stevo to fix the builds for us,  but as he said is 
unable to contribute right now.  The main issues (solved by hacks), currently 
are


  1.  Dependencies and transitive dependencies  are not being picked and copied 
to the `./lib` directory, where `/bin/mahout` and parts of the 
MahoutSparkContext look for them, to add to the class path.  So running either 
from the CLI or as a library, dependencies are not picked up.
 *   We used to use the mahout-experimental-xx.jar as a fat jar for this, 
though it was bloated with now deprecated MR stuff, and no longer packed.
  2.  `./bin/mahout` (and `compute-classpath.sh`) need to be revamped to ensure 
that they are picking up the correct classes.

w.r.t. to Java 8/7 issues, We did mandate Java 8+, and this required a few 
minor code changes to play nicely with Scala 2.11.  Mainly one class needed a 
JVM "Static" field, so i refactored that field out of the Class and into a 
companion object.  I wonder if this is what is giving you issues with Java 7.

I'd thought that Java 8 was mandated now, but may be thinking of maven 3.3.x.

Regardless Thank you very much for this.  This board is doing really doing well 
so far. and deserves accolades.

>
> 
> org.apache.mahout
> mahout-spark
> 14.1-SNAPSHOT
> 2.11
> 
This would be perfect IMO.


I can send you the commits that I am talking about.

As well, I saw that Trevor gave you a link to a filter.. I have one here with a 
bit more limited scope, which is open issues fixversion == 14.1.

To answer one question yes this was recently building, and releasing, with all 
of the tests passing (for a few modules, that we were focusing on).  after that 
i made some changes that broke it again..

the board with limited scope: 
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=348=detail=MAHOUT-2093


Thanks again for helping out.  we are really bad with poms, not so much from 
the ground up, as fixing some that are 10 years old, as Stevo mentioned, very 
quickly while working on several other things.

Thank you again for this. It is a great help, and once we get a good build, we 
can get back to doing work on the library itself.

I have some documents that i can provide if it will help explain the structure 
of the project, which is still kind of in flux.  E.g. I'd like to get the 
ViennaCL-OMP branch out of experimental, but there is much to clean up first.  
As well, I am on medical leave, and dont have much time on the computer these 
days.. have to budget my time.

I'll send you some (closed) PRs with notes and changes, if it helps.  lmk.

Thanks again, This is Huge.

Andy

From: Christofer Dutz 
Sent: Thursday, April 16, 2020 9:50 AM
To: dev@mahout.apache.org 
Subject: Re: Hi ... need some help?

Hi Trevor,

ok ... first of all ... the Mahout PMC is defining a "community maintained" 
library which is not maintained by the mahout PMC?!?!
I thought at Apache everything is about Community over code. So is a company 
driving the non-community stuff?

But back to your build issues:
I had a look and I too encountered these comments and remarks and sometimes 
patterns I recognized and could imagine why they were created.
Yes quite a bit of the build could be cleaned up and simplified a lot.

So how about I create a fork and try to do a cleanup of the build.
Usually I also leave comments about what I do as I hope I'll not be the only 
one maintaining a build and documenting things helps people feel more confident.

However in some cases I will have questions ... so would someone be available 
on Slack for quick questions?

Usually switching to another build system does solve some 

Re: Hi ... need some help?

[Andrew Palumbo]

Chris, Thank you so much for what you are doing,  This is Apache at its best.. 
I've been down and out with a serious Illness, Injury and other issues, which 
have seriously limited my Machine time.   I was pretty close to getting a good 
build, but it was hacky, and the method that you use to name the modules for 
both Scala versions, looks great.

We've always relied on Stevo to fix the builds for us,  but as he said is 
unable to contribute right now.  The main issues (solved by hacks), currently 
are


  1.  Dependencies and transitive dependencies  are not being picked and copied 
to the `./lib` directory, where `/bin/mahout` and parts of the 
MahoutSparkContext look for them, to add to the class path.  So running either 
from the CLI or as a library, dependencies are not picked up.
 *   We used to use the mahout-experimental-xx.jar as a fat jar for this, 
though it was bloated with now deprecated MR stuff, and no longer packed.
  2.  `./bin/mahout` (and `compute-classpath.sh`) need to be revamped to ensure 
that they are picking up the correct classes.

w.r.t. to Java 8/7 issues, We did mandate Java 8+, and this required a few 
minor code changes to play nicely with Scala 2.11.  Mainly one class needed a 
JVM "Static" field, so i refactored that field out of the Class and into a 
companion object.  I wonder if this is what is giving you issues with Java 7.

I'd thought that Java 8 was mandated now, but may be thinking of maven 3.3.x.

Regardless Thank you very much for this.  This board is doing really doing well 
so far. and deserves accolades.

>
> 
> org.apache.mahout
> mahout-spark
> 14.1-SNAPSHOT
> 2.11
> 
This would be perfect IMO.


I can send you the commits that I am talking about.

As well, I saw that Trevor gave you a link to a filter.. I have one here with a 
bit more limited scope, which is open issues fixversion == 14.1.

To answer one question yes this was recently building, and releasing, with all 
of the tests passing (for a few modules, that we were focusing on).  after that 
i made some changes that broke it again..

the board with limited scope: 
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=348=detail=MAHOUT-2093


Thanks again for helping out.  we are really bad with poms, not so much from 
the ground up, as fixing some that are 10 years old, as Stevo mentioned, very 
quickly while working on several other things.

Thank you again for this. It is a great help, and once we get a good build, we 
can get back to doing work on the library itself.

I have some documents that i can provide if it will help explain the structure 
of the project, which is still kind of in flux.  E.g. I'd like to get the 
ViennaCL-OMP branch out of experimental, but there is much to clean up first.  
As well, I am on medical leave, and dont have much time on the computer these 
days.. have to budget my time.

I'll send you some (closed) PRs with notes and changes, if it helps.  lmk.

Thanks again, This is Huge.

Andy

From: Christofer Dutz 
Sent: Thursday, April 16, 2020 9:50 AM
To: dev@mahout.apache.org 
Subject: Re: Hi ... need some help?

Hi Trevor,

ok ... first of all ... the Mahout PMC is defining a "community maintained" 
library which is not maintained by the mahout PMC?!?!
I thought at Apache everything is about Community over code. So is a company 
driving the non-community stuff?

But back to your build issues:
I had a look and I too encountered these comments and remarks and sometimes 
patterns I recognized and could imagine why they were created.
Yes quite a bit of the build could be cleaned up and simplified a lot.

So how about I create a fork and try to do a cleanup of the build.
Usually I also leave comments about what I do as I hope I'll not be the only 
one maintaining a build and documenting things helps people feel more confident.

However in some cases I will have questions ... so would someone be available 
on Slack for quick questions?

Usually switching to another build system does solve some problems ... mostly 
the reason to switch is that it solved the main problem that you are having 
with the old.
However you usually notice too late that you get yourself a lot of new 
problems. I remember doing some contract work for an insurance company and they 
were totally down Maven-road but then had to build something with SBT ... in 
the end I compiled the thing on my laptop, copied it to a USB stick and told 
the people what was on the stick and that I'll be having a coffee and will be 
back in 30 minutes. When I came back the sick wasn't at the same place and the 
build problem was "solved" ;-)

So I think it's quite good to stick to maven ... that is very mature, you can 
do almost everything you want with it and it integrates perfectly into the 
Apache infrastructure.

But that's just my opinion.

So if you want me to help, I'll be happy to be of assistance.


Chris



Am 16.04.20, 15:28 

[jira] [Created] (MAHOUT-2102) transitive and direct dependencies not being piched up in ~/lib by /bin/mahout

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2102:
--

 Summary: transitive and direct dependencies not being piched up in 
~/lib by /bin/mahout
 Key: MAHOUT-2102
 URL: https://issues.apache.org/jira/browse/MAHOUT-2102
 Project: Mahout
  Issue Type: Bug
Affects Versions: 14.1
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
 Fix For: 14.1


need to ensure dependencies are added to {{/lib/dependencies}} or packed into 
each module jar to be picked up bu {{./bin/mahout}}

https://github.com/apache/mahout/compare/master...andrewpalumbo:dep-copy-2?expand=1



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MAHOUT-2100) Transitive dependencies (Scopt e.g.) are not being picked on the classpath nor shipped in ./lib

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2100:
--

 Summary: Transitive dependencies (Scopt e.g.) are not being picked 
on the classpath nor shipped in ./lib
 Key: MAHOUT-2100
 URL: https://issues.apache.org/jira/browse/MAHOUT-2100
 Project: Mahout
  Issue Type: Bug
  Components: build
Affects Versions: 14.1
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
 Fix For: 14.1






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MAHOUT-2091) Apache Rat check fails: Too many files with unapproved license

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2091:
--

 Summary: Apache Rat check fails: Too many files with unapproved 
license
 Key: MAHOUT-2091
 URL: https://issues.apache.org/jira/browse/MAHOUT-2091
 Project: Mahout
  Issue Type: Bug
Affects Versions: 0.14.0
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
 Fix For: 14.1
 Attachments: rat.txt

running {{mvn clean apache-rat:check -e -DskipTests=true}} the build fails with

 
{code:java}
ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.13:check 
(default-cli) on project mahout: Too many files with unapproved license: 2 See 
RAT report in: /Users/colleenpalumbo/sandbox/mahout/target/rat.txt -> [Help 1]
 org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal 
org.apache.rat:apache-rat-plugin:0.13:check (default-cli) on project mahout: 
Too many files with unapproved license: 2 See RAT report in: 
/Users/colleenpalumbo/sandbox/mahout/target/rat.txt{code}
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MAHOUT-2090) RC4 - dixstribution module is not included in src distribution

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2090:
--

 Summary: RC4 - dixstribution module is not included in src 
distribution
 Key: MAHOUT-2090
 URL: https://issues.apache.org/jira/browse/MAHOUT-2090
 Project: Mahout
  Issue Type: Bug
Affects Versions: 14.1
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
 Fix For: 14.1






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MAHOUT-2089) generate sha-256 and sha-512 checksums for all release artifacts

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2089:
--

 Summary: generate sha-256 and sha-512 checksums for all release 
artifacts
 Key: MAHOUT-2089
 URL: https://issues.apache.org/jira/browse/MAHOUT-2089
 Project: Mahout
  Issue Type: Bug
  Components: build, release
Affects Versions: 0.14.0
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
 Fix For: 14.1


SHA-256 and sha-512 artifacts are not being generated during release 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MAHOUT-2088) update Apache Parent pom to latest version (34)

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2088:
--

 Summary: update Apache Parent pom to latest version (34)
 Key: MAHOUT-2088
 URL: https://issues.apache.org/jira/browse/MAHOUT-2088
 Project: Mahout
  Issue Type: Bug
  Components: build
Affects Versions: 0.14.0
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
 Fix For: 14.1


we are currently using Apache parent pom 18 which is from 2010:

http://maven.apache.org/pom/maven/

Upgrade to most recent pom: version 34.  This may alleviate some of our build 
issues?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[ANNOUNCE] Mahout 14.1 RC4

[Andrew Palumbo]

Fixing some typos and mistyping:

Hi All,

Ive finished RC4 of Apache Mahout 0.14.  Unfortunately Right off the bat, I can 
see an issue, which will require a new RC (only SHA-1) checksums have been 
included.  Apache requires at least sha256.  So I will -1 it myself.  I'd drop 
it and fix that now, but its late.

Please bear with me as this is a completely new structure of the project, which 
required almost a complete rewrite of all poms, as well as my first time acting 
as release engineer.

@PMC,  As I've already this RC is invalid,  I don't think that a vote is 
necessary, but if you all could take a look through this, and let me know if 
you see any issues, from trivial to major, it would be a great help in getting 
this release out the door finally.  I'm going to take a page out of Suneels 
book and ask for certain tasks that we need to get get RC5 down hopefully to 
just running tests.

resolver: [1] 
https://repository.apache.org/content/repositories/orgapachemahout-1061
artifacts: [2] 
https://repository.apache.org/content/repositories/orgapachemahout-1061/org/apache/mahout/

@rawkintrevo, I know that you needed binaries, 
could you please try to resolve artifacts that you need and give them a spin, 
and  if you have time check to see if the binary distribution. tar.gz contains 
all the necessary?

@pat could you please see if the 2.11 artifacts 
suit your specific needs (and work)?

@akm could you please  give a once over (or twice, or thrice over if you have 
time) and see if there are any gaping, obvious issues, especially with the 
binary and source release artifacts, as well could you please let me know about 
the new structure (releasing 2.11 and 2.12 in the same repo), and if it seems 
to work?

I'll have to concentrate my limited time, this week, on squashing these last 
couple of bugs.

@all please let me know if the structure seems to work.

If anyone has a quick script to verify the sigs it would  be useful but that's 
probably best saved for RC5.

As well I've used the naming scheme used by SBT:

   ${module_name}_{scala.major.minor}-{mahout version}

Please keep an eye out for any transpositions between ${scala.major.minor} and 
${mahout version}.

Both SBT and maven should have no issue resolving these artifacts (I tested 
maven only in an earlier RC).


It is  late, and I've done no testing other than running tests before i cut 
this RC, nor have i looked closely at any of the artifacts, so anything that 
seems out of the ordinary, Please take note of

There still are still issues with releasing that i have to fix, I.e  the 
maven-release-plugin is not functioning correctly (I believe, haven't fully 
tracked down the bug) these artifacts were .  its a new issue since RC3 and 
there have been very few changes since then, it may be as simple as some 
spacing in the pom (first few answers i found when searching.  The issue may be 
with the formatting of the XML itself.  I'll try to track this down as when I 
get back to it.  And as I mentioned, the release:perform goal failed, when run 
as:

 $ mvn clean package install -DskipTests -Papache-release 
release:perform

I will try to create a Jirass for these issues, (release failing and sha1) I'm 
not sure how much work I'll be able to do on this next week or over the 
weekend.  The release instructions have changed slightly and I will document 
these in the Jiras or on the PR with the new required ~/.m2/settings.xml.

I will be on and offline for the next few days, so @akm, if you want to release 
before i get back to this, I'll finish documenting the release steps in either 
the Jira, or the PR that I sent to you yesterday [3].

If we run these few smoke tests, and sanity checks (and as many more as anyone 
has time for), it will go a long way in raising the probability of RC5 being 
the final.

Of course, as always, any testing and feedback is welcomed and greatly  
appreciated from anyone on the list.

Thanks very much,

Andy


BTW:

It seems that the RC was not assigned a test number for the the artifacts [4]. 
e.g:


 org.apache.mahout
 mahout-core_2.11
 14.1


This should not be an issue, as there is no other version 14.1 in maven central 
or other repos, nor are there a scala 2.11 or 2.12 versions of anything [4].  
However if you've tried testing a 14.1 RC before, please remember to use the -U 
option when building, or clear out your ~/.m2 ./repository.

[1] https://repository.apache.org/content/repositories/orgapachemahout-1061
[2] 
https://repository.apache.org/content/repositories/orgapachemahout-1061/org/apache/mahout/
[3] 
https://github.com/apache/mahout/pull/384/commits/8794c42f910ed6fc3ac93d4081a88385520cc84b
[4] https://repository.apache.org/#stagingRepositories
[5] 
https://mvnrepository.com/artifact/org.apache.mahout/mahout-spark_2.10/0.13.0

[ANNOUNCE] Mahout 0.14 RC4 I

[Andrew Palumbo]

Hi All,

Ive finished RC4 of Apache Mahout 0.14.  Unfortunately Right off the bat, I can 
see an issue, which will require a new RC (only SHA-1) checksums have been 
included.  Apache requires at least sha256.  So I will -1 it myself.  I'd drop 
it and fix that now, but its late.

@mahout PMCs

Please bear with me as this is a completely new structure of the project, which 
required almost a complete rewrite of all poms, as well as my first time acting 
as release engineer.

@PMC,  As I've already this RC is invalid,  I don't think that a vote is 
necessary, but if you all could take a look through this, and let me know if 
you see any issues, from trivial to major, it would be a great help in getting 
this release out the door finally.  I'm going to take a page out of Suneels 
book and ask for certain tasks that we need to get get RC5 down hopefully to 
just running tests.

resolver: [1] 
https://repository.apache.org/content/repositories/orgapachemahout-1061
artifacts: [2] 
https://repository.apache.org/content/repositories/orgapachemahout-1061/org/apache/mahout/

@rawkintrevo, I know that you needed binaries, 
could you please try to resolve artifacts that you need and give them a spin, 
and  if you have time check to see if the binary distribution. tar.gz contains 
all the necessary?

@pat could you please see if the 2.11 artifacts 
suit your specific needs (and work)?

@akm could you please  give a once over (or twice, or thrice over if you have 
time) and see if there are any gaping, obvious issues, especially with the 
binary and source release artifacts, as well could you please let me know about 
the new structure (releasing 2.11 and 2.12 in the same repo), and if it seems 
to work?

I'll have to concentrate my limited time, this week, on squashing these last 
couple of bugs.

@all please let me know if the structure seems to work.

If anyone has a quick script to verify the sigs it would  be useful but that's 
probably best saved for RC5.

As well I've used the naming scheme used by SBT:

   ${module_name}_{scala.major.minor}-{mahout version}

Please keep an eye out for any transpositions between ${scala.major.minor} and 
${mahout version}.

Both SBT and maven should have no issue resolving these artifacts (I tested 
maven only in an earlier RC).


It is  late, and I've done no testing other than running tests before i cut 
this RC, nor have i looked closely at any of the artifacts, so anything that 
seems out of the ordinary, Please take note of

There still are still issues with releasing that i have to fix, I.e  the 
maven-release-plugin is not functioning correctly (I believe, haven't fully 
tracked down the bug) these artifacts were .  its a new issue since RC3 and 
there have been very few changes since then, it may be as simple as some 
spacing in the pom (first few answers i found when searching.  The issue may be 
with the formatting of the XML itself.  I'll try to track this down as when I 
get back to it.  And as I mentioned, the release:perform goal failed, when run 
as:

 $ mvn clean package install -DskipTests -Papache-release 
release:perform

I will try to create a Jirass for these issues, (release failing and sha1) I'm 
not sure how much work I'll be able to do on this next week or over the 
weekend.  The release instructions have changed slightly and I will document 
these in the Jiras or on the PR with the new required ~/.m2/settings.xml.

I will be on and offline for the next few days, so @akm, if you want to release 
before i get back to this, I'll finish documenting the release steps in either 
the Jira, or the PR that I sent to you yesterday [3].

If we run these few smoke tests, and sanity checks (and as many more as anyone 
has time for), it will go a long way in raising the probability of RC5 being 
the final.

Of course, as always, any testing and feedback is welcomed and greatly  
appreciated from anyone on the list.

Thanks very much,

Andy


BTW:

It seems that the RC was not assigned a test number for the the artifacts [. 
e.g:


 org.apache.mahout
 mahout-core_2.11
 14.1


This should not be an issue, as there is no other version 14.1 in maven central 
or other repos, nor are there a scala 2.11 or 2.12 versions of anything.  
However if you've tried testing a 14.1 RC before, please remember to use the -U 
option when building, or clear out your ~/.m2 ./repository.

[1] https://repository.apache.org/content/repositories/orgapachemahout-1061
[2] 
https://repository.apache.org/content/repositories/orgapachemahout-1061/org/apache/mahout/
[3] 
https://github.com/apache/mahout/pull/384/commits/8794c42f910ed6fc3ac93d4081a88385520cc84b
[4] https://repository.apache.org/#stagingRepositories
[5] 
https://mvnrepository.com/artifact/org.apache.mahout/mahout-spark_2.10/0.13.0

Mahout Slack Forwards Messages on the #0.14.0-release to dev@mahout.apache.org

[Andrew Palumbo]

Hello All,

We have set up Mahout's slack space to forward directly to 
dev@mahout.apache.org.  We will now be able to plan publicly on slack.  This a 
bi-directional connection, all messages to dev@mahout.apache.org will show up 
in Slack.  No one will be left out of planning.

IRC, Slack, HipChat and other messaging platforms have always been as they are 
and are now essential tools optimal and effective distributed software 
development.

Surprisingly, there are not a huge amount of freely available methods to ways 
to connect slack to there outside universe or at least were not when i started 
this back burner project last year.

At the time, there there was essential : The MailClark APP and Slack 
Web-hooks[1]. Slack web-hooks require an outside HTML/PHP server and take 
reasonably easy configuration, available on GitHub.  This would have been 
relatively easy and required only a few minutes of PHP copy pasta, but also 
would have meant setting up and maintaining an email server specifically for 
this.

We went with a free version of the Mail Clark app.  The free version allows a 
relatively easy configuration for a single channel in slack to forward each 
message sent to d...@your-project.mahout.org during a planning session without 
interruption.  each message will come from the bot's email address, with the 
Full Name name of the person sending it and the channel name in the subject.  
Slack threads are maintained as email threads on the dev@ list.  As i 
mentioned, the connection can configured to be be bi-directional.

In some cases It's been easier to have conversations, on other channels, and 
then copy the entire day's messages in to either an email or just pasting it 
directly into the connected account.  As i mentioned, the connection is 
bi-directional, so those not using slack can participate via email.

This allows us to either record planning sessions or project issues verbatim in 
Slack, with really no overhead.

This has helped us to keep the efficiency of slack, while maintaining 
compliance with ASF public e-mail list planning requirements.

Please let me know if a write-up of MailClark integration would be useful.  It 
can be done for a single channel in about 10 minutes (I struggled on and off 
for a while to get it done in a different manner, and found the single channel 
method to be easiest.)  The interface has matured a bit over the past year, and 
is now relatively intuitive, but also not so much.

I am happy to provide a write up of the steps to connect if it would be useful 
for anyone.


If it didn't happen on list, it didn't happen.


Please let me know if anyone is interested.


--Andy




[ 1 ] 
https://slack.com/help/articles/115005265063-incoming-webhooks-for-slack#messag[e-options.
 

[ 2 ] https://mailclark.ai/
[https://mailclark.ai/static/img/logos/slack-avatar.png]
MailClark: Smart Shared Inbox Within Slack & Microsoft 
Teams
Whether it be a Facebook Page, Twitter account or several email addresses 
(Outlook, Gmail, etc.) you and your team can access, interact and monitor 
everything directly from the workspace.
mailclark.ai



[https://a.slack-edge.com/80588/marketing/img/meta/slack_hash_256.png]
Incoming WebHooks for Slack | 
Slack
Using Slack From channels to search — learn how Slack works from top to bottom! 
Workspace Administration Want to learn more about setting up your team? Look no 
further! Extras Supplemental Slack info for you and your team. Changelog If 
you’re curious about what’s new in Slack — and what’s changed — you’re in the 
right place. Slack Certification Your Profile & Preferences Adjust ...
slack.com

[jira] [Created] (MAHOUT-2086) change jar naming scheme to the corredct convention

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2086:
--

 Summary: change jar naming scheme to the corredct convention
 Key: MAHOUT-2086
 URL: https://issues.apache.org/jira/browse/MAHOUT-2086
 Project: Mahout
  Issue Type: Bug
Affects Versions: 14.1
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
 Fix For: 14.1


currently we build jars as in the {{$MAHOUT_HOME/module/target






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MAHOUT-2085) Add scm tag for each pom.xml per asf release Guidelines

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2085:
--

 Summary: Add scm tag for each pom.xml per asf release Guidelines
 Key: MAHOUT-2085
 URL: https://issues.apache.org/jira/browse/MAHOUT-2085
 Project: Mahout
  Issue Type: Bug
Affects Versions: 0.14.0
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
 Fix For: 14.1


https://www.apache.org/dev/publishing-maven-artifacts.html 

Is pretty adament about having a SCM tag in each pom.  Since our project has a 
pretty complicated structure, adding this tag back in to stay with guidelines.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Small refactor for RC4

[Andrew Palumbo]

Hey all,
While fixing a few missing artifacts from the source distribution, I finally 
found the apache-release profile instructions [1].  It clarifies the Apache 
naming and prefrred project structure.. I have 
https://github.com/apache/mahout/pull/389 open to work on it a relatively minor 
refactor to get us in line with the suggested structure.



Essentially the #389 refactors the


   distribution_${scala.compat.version}/


module into a single dist/


module with no scala version appended (both 2.11 and 2.12 distribution 
artifacts). this due to some automatic mirroring script which picks up 
distribution releases from the dist/release folder [1] on the apache server and 
copies it to mirrors.. Still need to ensure that the artifacts are signed 
properly when uploaded. [1] is possibly out of date however i did see this 
mentioned on other projects.. by using a dist/release directory for the full 
tarballs, Mahout 14.1 bin and src distributions will be downloadable from 
https://dist.apache.org/repos/dist/.


This will give us a final structure of:

├── bin
 | |___mahout
 |
├── community
 |   \__mahout-spark-cli-drivers_2.11
 |   \__mahout-spark-cli-drivers_2.12
 |
├── core_2.11
 |core_2.12
 |
├── dist
 |  \___release
 |   |___apache-mahout-14.1_2.11-bin.tar.gz
 |   |___apache-mahout-14.1_2.11-bin.tar.gz.asc
 |   |___apache-mahout-14.1_2.11-bin.bz2
 |   |___apache-mahout-14.1_2.11-bin.bz2.asc
 |   |___apache-mahout-14.1_2.11-src.tar.gz
 |   |___apache-mahout-14.1_2.11-src.tar.gz.asc
 |   |___apache-mahout-14.1_2.11-src.bz2
 |   |___apache-mahout-14.1_2.11-src.bz2.asc
 |
├── engine
 | \___mahout-hdfs_2.11
 | \___mahout-hdfs_2.12
 |
 | \___mahout-spark_2.11
 | \___mahout-spark_2.12
 |
├── examples
 | \___bin
 | |___current_examples.sh
 |  \___resources
 |
├── lib
 |  \___all_mahout-14.1_2.11.jars
 |  \___all_mahout-14.1_2.12.jars
 |
 |── website
 |
├── pom.xml



That will be essentially the tree that will be available on nexus.  There is no 
clear guide on how best to release a cross compiled build, but i think that 
this should be fine.  previously artifacts have shown in nexus correctly:

To test, I have used:


org.apache.mahout
mahout-core_${scala.compat.version}
14.1



org.apache.mahout
mahout-hdfs_${scala.compat.version}
14.1



org.apache.mahout
mahout-spark_${scala.compat.version}
14.1
tests



org.apache.mahout
mahout-spark_${scala.compat.version}
14.1



org.apache.mahout

mahout-spark-cli-drivers_${scala.compat.version}
14.1


When the artifacts were staged for RC3.  Per [3], It seems that we must have an 
SCM tag on each pom:

  Staging a release (1) preparing poms (3):

  (3) "Verify that all pom.xml files have an SCM definition." [3]


I'd added these in previously but thought I'd had an issue.  Adding them in now 
before RC4.

I wanted to outline some of the changes that might seem odd.  Any 
suggestions/comments/criticisms are welcome.

Thanks all,

I hope to have RC4 out shortly.


[1] https://www.apache.org/dev/release-publishing#distribution

[2] https://www.apache.org/dev/publishing-maven-artifacts.html#publish-snapshot

[3] https://www.apache.org/dev/publishing-maven-artifacts.html

[jira] [Created] (MAHOUT-2084) a default source distribution for is being created under

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2084:
--

 Summary: a default source distribution for   is being created under
 Key: MAHOUT-2084
 URL: https://issues.apache.org/jira/browse/MAHOUT-2084
 Project: Mahout
  Issue Type: Bug
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo


make this behavior stop



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MAHOUT-2083) Release sources contain pom.XmlBackup and are mising engine and hdfs submodules

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2083:
--

 Summary: Release sources contain pom.XmlBackup and are mising 
engine and hdfs submodules
 Key: MAHOUT-2083
 URL: https://issues.apache.org/jira/browse/MAHOUT-2083
 Project: Mahout
  Issue Type: Bug
  Components: release
Affects Versions: 14.1
Reporter: Trevor Grant
Assignee: Andrew Palumbo
 Fix For: 14.1






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

[Andrew Palumbo]

Thanks Trevor- I see.  Looks like the backup poms are included also from during 
the release.   that is odd.  I will also try to fix the default distribution 
bug that i noticed. since Its the middle of the week.

OK dropping RC new one to come soon.

-andy



From: Trevor Grant 
Sent: Tuesday, January 21, 2020 5:24 PM
To: Mahout Dev List 
Cc: Andrew Musselman 
Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

Hey,

I _think_ I'm -1

I went to build src (mvn clean install) on the 2.12 src and got an error:

```
[ERROR] Child module
/home/rawkintrevo/mahout-rc-test/engine/spark/pom.xml of
/home/rawkintrevo/mahout-rc-test/pom.xml does not exist
[ERROR] Child module
/home/rawkintrevo/mahout-rc-test/engine/hdfs/pom.xml of
/home/rawkintrevo/mahout-rc-test/pom.xml does not exist
```

I checked, and sure enough `engine/spark` and `engine/hdfs` are empty. Same
is true in 2.11 distribution.

Also- this is a very very confusing build.

The two source tarballs I was testing were:
https://repository.apache.org/content/repositories/orgapachemahout-1060/org/apache/mahout/distribution_2.11/14.1/distribution_2.11-14.1-src.tar.gz
https://repository.apache.org/content/repositories/orgapachemahout-1060/org/apache/mahout/distribution_2.12/14.1/distribution_2.12-14.1-src.tar.gz

If these are the wrong ones, I'll rescind my -1






On Tue, Jan 14, 2020 at 3:19 PM Andrew Palumbo  wrote:

> This is a call for volunteers as well.  Anyone may test the Release
> candidate;  the more eyes the better.
>
> Changed subject Line.
>
> --Andy
>
> 
> From: Andrew Musselman 
> Sent: Tuesday, January 14, 2020 1:43 PM
> To: dev@mahout.apache.org 
> Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for testing and vote
>
> I will have time Thursday afternoon; thanks Andy
>
> On Tue, Jan 14, 2020 at 05:52 Trevor Grant 
> wrote:
>
> > I can test tomorrow.
> >
> > On Tue, Jan 14, 2020 at 1:41 AM Andrew Palumbo 
> > wrote:
> >
> > > Hey all,
> > > I've uploaded RC3 to nexus for scala 2.11 and scala 2.12.  I already
> have
> > > seen a small issue while building the RC.  All artifacts are available
> in
> > > staging:
> > >
> > >
> > >
> >
> https://repository.apache.org/service/local/repositories/orgapachemahout-1060/content/org/apache/mahout/
> > >
> > > We should test this release well, as it is a heavy refactor, We still
> > have
> > > not tested /bin/mahout.
> > >
> > > So we need three binding +1s for this release.
> > >
> > > For testing, lease do the following:
> > >  1. Download build and test the src archive.
> > >  2. Run a job CCO e.g which uses the CLI.
> > >  3. run on a cluster/pseudo or otherwise.
> > >  4. Verify Sigs and hashes.
> > >
> > > I noticed that many people are busy this week.  I am wondering if we
> > > can/should leave the Vote open longer than 72hrs.
> > >
> > > Or if we can work out all trivial issues, like naming and this extra
> > > source distribution, and I'll build RC4.
> > >
> > >
> > > Thank you,
> > >
> > > Andy
> > >
> > >
> >
>

Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

[Andrew Palumbo]

no rush here.. Andrew is busy till weekend, it looks like and Joe confirmed. 
the minimum..  I have the RC process for this release down to about 45 mins so 
can easily cut a new one any time anyways.

From: Trevor Grant 
Sent: Tuesday, January 21, 2020 5:54 AM
To: Mahout Dev List 
Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

I think 72 hours is the _minimum_.

I came back late last night so I could have today for laundry, cleaning,
and other catch-up-on-life stuff, which would include testing the RC.

Tg


On Jan 21, 2020 12:17 AM, "Andrew Musselman" 
wrote:

I think 72 hours is a guideline but can check; my apologies but I also got
too busy with work and family to take a good look.

If we have the right URL for artifacts we can shoot for a vote this weekend.

Thanks for running the release Andy!


On Mon, Jan 20, 2020 at 21:35 Andrew Palumbo  wrote:

> As a reminder to all.  I cut this RC at a bad point for most to test.
> (Rawkintrevo had a vacation and others mentioned offline being very busy.
> We discussed offline leaving the vote open for more than 72 hours. but I
am
> happy to close it and re open when it is not Announced bad timing.  or in
> voilation of Apache release rules (might be)...  Please let me know.
>
> Maybe we can co-ordinate at next weekly meeting when Trevor's back? It
> seemed like a good amount were ready to participate in weelky meetings
last
> week.
>
> (I also supplied incorrect artifact name in release.. My first time acting
> as release manager.)
>
> Happy to coordinate a better time for testing and cut a new RC if need be.
>
> Whatever's good.
>
> --Andy
> --
> *From:* Andrew Palumbo 
> *Sent:* Saturday, January 18, 2020 7:19 PM
>
> *To:* Andrew Musselman ; dev@mahout.apache.org
> 
> *Subject:* Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

>
> Resolved artifacts for a test application.  Built and tested.  There is a
> duplicate source release however, I believe that this is trivial and can
be
> dealt with in a readme or not at all (it is a default artifact from the
> maven release plugin).
>
> +1 (binding)
>
> On Jan 16, 2020 10:50 PM, Andrew Palumbo  wrote:
>
> I looked back over the release docs, and the link I was supposed to send
> out  out was this:
> https://repository.apache.org/content/repositories/orgapachemahout-1060/
> <
https://slack-redir.net/link?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachemahout-1060%2F=3
>
>
> I was able to successfully resolve all artifacts  with this link in a
> spark  application.
>
> please use this URL for to test the staged artifacts.
>
> Thank you,
>
> --andy
> --
> *From:* Andrew Palumbo 
> *Sent:* Wednesday, January 15, 2020 5:01 PM
> *To:* Andrew Musselman ; dev@mahout.apache.org
> 
> *Subject:* Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

>
> Apologies, I'd not closed the staging repository, and sent out the wrong
> link In the initial email. the correct staging link is here:
>
>
>
https://repository.apache.org/content/repositories/orgapachemahout-1060/org/apache/mahout/
<
https://slack-redir.net/link?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachemahout-1060%2Forg%2Fapache%2Fmahout%2F=3
> <
https://repository.apache.org/content/repositories/orgapachemahout-1060/org/apache/mahout/%3Chttps://slack-redir.net/link?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachemahout-1060%2Forg%2Fapache%2Fmahout%2F=3
>

> >
>
> --andy
> 
> From: Andrew Palumbo 
> Sent: Tuesday, January 14, 2020 4:18 PM
> To: Andrew Musselman ; dev@mahout.apache.org <
> dev@mahout.apache.org>
> Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote
>
> This is a call for volunteers as well.  Anyone may test the Release
> candidate;  the more eyes the better.
>
> Changed subject Line.
>
> --Andy
>
> 
> From: Andrew Musselman 
> Sent: Tuesday, January 14, 2020 1:43 PM
> To: dev@mahout.apache.org 
> Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for testing and vote
>
> I will have time Thursday afternoon; thanks Andy
>
> On Tue, Jan 14, 2020 at 05:52 Trevor Grant 
> wrote:
>
> > I can test tomorrow.
> >
> > On Tue, Jan 14, 2020 at 1:41 AM Andrew Palumbo 
> > wrote:
> >
> > > Hey all,
> > > I've uploaded RC3 to nexus for scala 2.11 and scala 2.12.  I already
> have
> > > seen a small issue while building the RC.  All artifacts are available
> in
> > > staging:
> > >
> > >
> > >

Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

[Andrew Palumbo]

Ah cool perfect-  thanks Joe.  --  Yes this is the document that i was thinking 
about so we're good It looks like.
So we can take our time on it.


Thanks alot!

--andy


From: Joey Frazee 
Sent: Tuesday, January 21, 2020 12:13 PM
To: Mahout Dev List ; dev@mahout.apache.org 

Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

I’ve seen this come up on a lot of lists. I wonder if it needs clarified. 
Release policy says "Release votes SHOULD remain open for *at least* 72 hours” 
[1] (emphasis on 'at least' mine). So going longer is 100% OK, and going 
shorter could even be justified in weird circumstances (security issue?).

Also, I don’t think any smaller projects interpret not passing in 72 hours as 
meaning the vote failed.

-joey

 1. https://apache.org/legal/release-policy.html
On Jan 21, 2020, 4:55 AM -0600, Trevor Grant , wrote:
> I think 72 hours is the _minimum_.
>
> I came back late last night so I could have today for laundry, cleaning,
> and other catch-up-on-life stuff, which would include testing the RC.
>
> Tg
>
>
> On Jan 21, 2020 12:17 AM, "Andrew Musselman" 
> wrote:
>
> I think 72 hours is a guideline but can check; my apologies but I also got
> too busy with work and family to take a good look.
>
> If we have the right URL for artifacts we can shoot for a vote this weekend.
>
> Thanks for running the release Andy!
>
>
> On Mon, Jan 20, 2020 at 21:35 Andrew Palumbo  wrote:
>
> > As a reminder to all. I cut this RC at a bad point for most to test.
> > (Rawkintrevo had a vacation and others mentioned offline being very busy.
> > We discussed offline leaving the vote open for more than 72 hours. but I
> am
> > happy to close it and re open when it is not Announced bad timing. or in
> > voilation of Apache release rules (might be)... Please let me know.
> >
> > Maybe we can co-ordinate at next weekly meeting when Trevor's back? It
> > seemed like a good amount were ready to participate in weelky meetings
> last
> > week.
> >
> > (I also supplied incorrect artifact name in release.. My first time acting
> > as release manager.)
> >
> > Happy to coordinate a better time for testing and cut a new RC if need be.
> >
> > Whatever's good.
> >
> > --Andy
> > --
> > *From:* Andrew Palumbo 
> > *Sent:* Saturday, January 18, 2020 7:19 PM
> >
> > *To:* Andrew Musselman ; dev@mahout.apache.org
> > 
> > *Subject:* Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote
>
> >
> > Resolved artifacts for a test application. Built and tested. There is a
> > duplicate source release however, I believe that this is trivial and can
> be
> > dealt with in a readme or not at all (it is a default artifact from the
> > maven release plugin).
> >
> > +1 (binding)
> >
> > On Jan 16, 2020 10:50 PM, Andrew Palumbo  wrote:
> >
> > I looked back over the release docs, and the link I was supposed to send
> > out out was this:
> > https://repository.apache.org/content/repositories/orgapachemahout-1060/
> > <
> https://slack-redir.net/link?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachemahout-1060%2F=3
> >
> >
> > I was able to successfully resolve all artifacts with this link in a
> > spark application.
> >
> > please use this URL for to test the staged artifacts.
> >
> > Thank you,
> >
> > --andy
> > --
> > *From:* Andrew Palumbo 
> > *Sent:* Wednesday, January 15, 2020 5:01 PM
> > *To:* Andrew Musselman ; dev@mahout.apache.org
> > 
> > *Subject:* Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote
>
> >
> > Apologies, I'd not closed the staging repository, and sent out the wrong
> > link In the initial email. the correct staging link is here:
> >
> >
> >
> https://repository.apache.org/content/repositories/orgapachemahout-1060/org/apache/mahout/
> <
> https://slack-redir.net/link?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachemahout-1060%2Forg%2Fapache%2Fmahout%2F=3
> > <
> https://repository.apache.org/content/repositories/orgapachemahout-1060/org/apache/mahout/%3Chttps://slack-redir.net/link?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachemahout-1060%2Forg%2Fapache%2Fmahout%2F=3
> >
>
> > >
> >
> > --andy
> > 
> > From: Andrew Palumbo 
> > Sent: Tuesday, January 14, 2020 4:18 PM
> > To: Andrew Musselman ; dev@mahout.apache.org <
> > dev@mahout.

Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

[Andrew Palumbo]

No Prob as far as running the release. Yeah sorry about the timing you told me 
you were busy too, Andrew.  I thought we were going to have more issues to 
clean up.  From this run though I think we have a viable candidate.  Since so 
much of the overhaul of the poms was in the actual releasing I was not paying 
so much attention to peoples schedules. Next time will make sure to 
coo-ordinate.

I think that my pom skills may have actually diminished after working with this 
release :) It is slightly hacky, but i think that we should consider any 
further overhaul as a team, I already had to make some changes to the 
~/.m2/settings.xml of the release master, documented on the mvn site, this may 
be more than we want.  I have the steps documented, and will update the 
how-to-release.md page.  and sart a jira for the new settings as well.

tl;dr, any further work on poms IMO should be a new design plan.

Release artifact URL (scala 2.11, 2.12): 
https://repository.apache.org/content/repositories/orgapachemahout-1060/<https://slack-redir.net/link?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachemahout-1060%2F=3>

--andy

From: Andrew Musselman 
Sent: Tuesday, January 21, 2020 1:17 AM
To: Andrew Palumbo 
Cc: dev@mahout.apache.org 
Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

I think 72 hours is a guideline but can check; my apologies but I also got too 
busy with work and family to take a good look.

If we have the right URL for artifacts we can shoot for a vote this weekend.

Thanks for running the release Andy!

On Mon, Jan 20, 2020 at 21:35 Andrew Palumbo 
mailto:ap@outlook.com>> wrote:
As a reminder to all.  I cut this RC at a bad point for most to test. 
(Rawkintrevo had a vacation and others mentioned offline being very busy.   We 
discussed offline leaving the vote open for more than 72 hours. but I am happy 
to close it and re open when it is not Announced bad timing.  or in voilation 
of Apache release rules (might be)...  Please let me know.

Maybe we can co-ordinate at next weekly meeting when Trevor's back? It seemed 
like a good amount were ready to participate in weelky meetings last week.

(I also supplied incorrect artifact name in release.. My first time acting as 
release manager.)

Happy to coordinate a better time for testing and cut a new RC if need be.

Whatever's good.

--Andy
____
From: Andrew Palumbo mailto:ap@outlook.com>>
Sent: Saturday, January 18, 2020 7:19 PM

To: Andrew Musselman 
mailto:andrew.mussel...@gmail.com>>; 
dev@mahout.apache.org<mailto:dev@mahout.apache.org> 
mailto:dev@mahout.apache.org>>
Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

Resolved artifacts for a test application.  Built and tested.  There is a 
duplicate source release however, I believe that this is trivial and can be 
dealt with in a readme or not at all (it is a default artifact from the maven 
release plugin).

+1 (binding)

On Jan 16, 2020 10:50 PM, Andrew Palumbo 
mailto:ap@outlook.com>> wrote:
I looked back over the release docs, and the link I was supposed to send out  
out was this: 
https://repository.apache.org/content/repositories/orgapachemahout-1060/<https://slack-redir.net/link?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachemahout-1060%2F=3>

I was able to successfully resolve all artifacts  with this link in a spark  
application.

please use this URL for to test the staged artifacts.

Thank you,

--andy
____
From: Andrew Palumbo mailto:ap@outlook.com>>
Sent: Wednesday, January 15, 2020 5:01 PM
To: Andrew Musselman 
mailto:andrew.mussel...@gmail.com>>; 
dev@mahout.apache.org<mailto:dev@mahout.apache.org> 
mailto:dev@mahout.apache.org>>
Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

Apologies, I'd not closed the staging repository, and sent out the wrong link 
In the initial email. the correct staging link is here:

https://repository.apache.org/content/repositories/orgapachemahout-1060/org/apache/mahout/<https://slack-redir.net/link?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachemahout-1060%2Forg%2Fapache%2Fmahout%2F=3<https://repository.apache.org/content/repositories/orgapachemahout-1060/org/apache/mahout/%3Chttps://slack-redir.net/link?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachemahout-1060%2Forg%2Fapache%2Fmahout%2F=3>>

--andy

From: Andrew Palumbo mailto:ap@outlook.com>>
Sent: Tuesday, January 14, 2020 4:18 PM
To: Andrew Musselman 
mailto:andrew.mussel...@gmail.com>>; 
dev@mahout.apache.org<mailto:dev@mahout.apache.org> 
mailto:dev@mahout.apache.org>>
Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

This is a call for volunteers

Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

[Andrew Palumbo]

As a reminder to all.  I cut this RC at a bad point for most to test. 
(Rawkintrevo had a vacation and others mentioned offline being very busy.   We 
discussed offline leaving the vote open for more than 72 hours. but I am happy 
to close it and re open when it is not Announced bad timing.  or in voilation 
of Apache release rules (might be)...  Please let me know.

Maybe we can co-ordinate at next weekly meeting when Trevor's back? It seemed 
like a good amount were ready to participate in weelky meetings last week.

(I also supplied incorrect artifact name in release.. My first time acting as 
release manager.)

Happy to coordinate a better time for testing and cut a new RC if need be.

Whatever's good.

--Andy

From: Andrew Palumbo 
Sent: Saturday, January 18, 2020 7:19 PM
To: Andrew Musselman ; dev@mahout.apache.org 

Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

Resolved artifacts for a test application.  Built and tested.  There is a 
duplicate source release however, I believe that this is trivial and can be 
dealt with in a readme or not at all (it is a default artifact from the maven 
release plugin).

+1 (binding)

On Jan 16, 2020 10:50 PM, Andrew Palumbo  wrote:
I looked back over the release docs, and the link I was supposed to send out  
out was this: 
https://repository.apache.org/content/repositories/orgapachemahout-1060/<https://slack-redir.net/link?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachemahout-1060%2F=3>

I was able to successfully resolve all artifacts  with this link in a spark  
application.

please use this URL for to test the staged artifacts.

Thank you,

--andy

From: Andrew Palumbo 
Sent: Wednesday, January 15, 2020 5:01 PM
To: Andrew Musselman ; dev@mahout.apache.org 

Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

Apologies, I'd not closed the staging repository, and sent out the wrong link 
In the initial email. the correct staging link is here:

https://repository.apache.org/content/repositories/orgapachemahout-1060/org/apache/mahout/<https://slack-redir.net/link?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachemahout-1060%2Forg%2Fapache%2Fmahout%2F=3>

--andy
____
From: Andrew Palumbo 
Sent: Tuesday, January 14, 2020 4:18 PM
To: Andrew Musselman ; dev@mahout.apache.org 

Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

This is a call for volunteers as well.  Anyone may test the Release candidate;  
the more eyes the better.

Changed subject Line.

--Andy


From: Andrew Musselman 
Sent: Tuesday, January 14, 2020 1:43 PM
To: dev@mahout.apache.org 
Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for testing and vote

I will have time Thursday afternoon; thanks Andy

On Tue, Jan 14, 2020 at 05:52 Trevor Grant  wrote:

> I can test tomorrow.
>
> On Tue, Jan 14, 2020 at 1:41 AM Andrew Palumbo 
> wrote:
>
> > Hey all,
> > I've uploaded RC3 to nexus for scala 2.11 and scala 2.12.  I already have
> > seen a small issue while building the RC.  All artifacts are available in
> > staging:
> >
> >
> >
> https://repository.apache.org/service/local/repositories/orgapachemahout-1060/content/org/apache/mahout/
> >
> > We should test this release well, as it is a heavy refactor, We still
> have
> > not tested /bin/mahout.
> >
> > So we need three binding +1s for this release.
> >
> > For testing, lease do the following:
> >  1. Download build and test the src archive.
> >  2. Run a job CCO e.g which uses the CLI.
> >  3. run on a cluster/pseudo or otherwise.
> >  4. Verify Sigs and hashes.
> >
> > I noticed that many people are busy this week.  I am wondering if we
> > can/should leave the Vote open longer than 72hrs.
> >
> > Or if we can work out all trivial issues, like naming and this extra
> > source distribution, and I'll build RC4.
> >
> >
> > Thank you,
> >
> > Andy
> >
> >
>

Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

[Andrew Palumbo]

Resolved artifacts for a test application.  Built and tested.  There is a 
duplicate source release however, I believe that this is trivial and can be 
dealt with in a readme or not at all (it is a default artifact from the maven 
release plugin).

+1 (binding)

On Jan 16, 2020 10:50 PM, Andrew Palumbo  wrote:
I looked back over the release docs, and the link I was supposed to send out  
out was this: 
https://repository.apache.org/content/repositories/orgapachemahout-1060/<https://slack-redir.net/link?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachemahout-1060%2F=3>

I was able to successfully resolve all artifacts  with this link in a spark  
application.

please use this URL for to test the staged artifacts.

Thank you,

--andy

From: Andrew Palumbo 
Sent: Wednesday, January 15, 2020 5:01 PM
To: Andrew Musselman ; dev@mahout.apache.org 

Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

Apologies, I'd not closed the staging repository, and sent out the wrong link 
In the initial email. the correct staging link is here:

https://repository.apache.org/content/repositories/orgapachemahout-1060/org/apache/mahout/<https://slack-redir.net/link?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachemahout-1060%2Forg%2Fapache%2Fmahout%2F=3>

--andy
____
From: Andrew Palumbo 
Sent: Tuesday, January 14, 2020 4:18 PM
To: Andrew Musselman ; dev@mahout.apache.org 

Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

This is a call for volunteers as well.  Anyone may test the Release candidate;  
the more eyes the better.

Changed subject Line.

--Andy


From: Andrew Musselman 
Sent: Tuesday, January 14, 2020 1:43 PM
To: dev@mahout.apache.org 
Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for testing and vote

I will have time Thursday afternoon; thanks Andy

On Tue, Jan 14, 2020 at 05:52 Trevor Grant  wrote:

> I can test tomorrow.
>
> On Tue, Jan 14, 2020 at 1:41 AM Andrew Palumbo 
> wrote:
>
> > Hey all,
> > I've uploaded RC3 to nexus for scala 2.11 and scala 2.12.  I already have
> > seen a small issue while building the RC.  All artifacts are available in
> > staging:
> >
> >
> >
> https://repository.apache.org/service/local/repositories/orgapachemahout-1060/content/org/apache/mahout/
> >
> > We should test this release well, as it is a heavy refactor, We still
> have
> > not tested /bin/mahout.
> >
> > So we need three binding +1s for this release.
> >
> > For testing, lease do the following:
> >  1. Download build and test the src archive.
> >  2. Run a job CCO e.g which uses the CLI.
> >  3. run on a cluster/pseudo or otherwise.
> >  4. Verify Sigs and hashes.
> >
> > I noticed that many people are busy this week.  I am wondering if we
> > can/should leave the Vote open longer than 72hrs.
> >
> > Or if we can work out all trivial issues, like naming and this extra
> > source distribution, and I'll build RC4.
> >
> >
> > Thank you,
> >
> > Andy
> >
> >
>

Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

[Andrew Palumbo]

I looked back over the release docs, and the link I was supposed to send out  
out was this: 
https://repository.apache.org/content/repositories/orgapachemahout-1060/<https://slack-redir.net/link?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachemahout-1060%2F=3>

I was able to successfully resolve all artifacts  with this link in a spark  
application.

please use this URL for to test the staged artifacts.

Thank you,

--andy

From: Andrew Palumbo 
Sent: Wednesday, January 15, 2020 5:01 PM
To: Andrew Musselman ; dev@mahout.apache.org 

Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

Apologies, I'd not closed the staging repository, and sent out the wrong link 
In the initial email. the correct staging link is here:

https://repository.apache.org/content/repositories/orgapachemahout-1060/org/apache/mahout/<https://slack-redir.net/link?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachemahout-1060%2Forg%2Fapache%2Fmahout%2F=3>

--andy
____
From: Andrew Palumbo 
Sent: Tuesday, January 14, 2020 4:18 PM
To: Andrew Musselman ; dev@mahout.apache.org 

Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

This is a call for volunteers as well.  Anyone may test the Release candidate;  
the more eyes the better.

Changed subject Line.

--Andy


From: Andrew Musselman 
Sent: Tuesday, January 14, 2020 1:43 PM
To: dev@mahout.apache.org 
Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for testing and vote

I will have time Thursday afternoon; thanks Andy

On Tue, Jan 14, 2020 at 05:52 Trevor Grant  wrote:

> I can test tomorrow.
>
> On Tue, Jan 14, 2020 at 1:41 AM Andrew Palumbo 
> wrote:
>
> > Hey all,
> > I've uploaded RC3 to nexus for scala 2.11 and scala 2.12.  I already have
> > seen a small issue while building the RC.  All artifacts are available in
> > staging:
> >
> >
> >
> https://repository.apache.org/service/local/repositories/orgapachemahout-1060/content/org/apache/mahout/
> >
> > We should test this release well, as it is a heavy refactor, We still
> have
> > not tested /bin/mahout.
> >
> > So we need three binding +1s for this release.
> >
> > For testing, lease do the following:
> >  1. Download build and test the src archive.
> >  2. Run a job CCO e.g which uses the CLI.
> >  3. run on a cluster/pseudo or otherwise.
> >  4. Verify Sigs and hashes.
> >
> > I noticed that many people are busy this week.  I am wondering if we
> > can/should leave the Vote open longer than 72hrs.
> >
> > Or if we can work out all trivial issues, like naming and this extra
> > source distribution, and I'll build RC4.
> >
> >
> > Thank you,
> >
> > Andy
> >
> >
>

Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

[Andrew Palumbo]

Apologies, I'd not closed the staging repository, and sent out the wrong link 
In the initial email. the correct staging link is here:

https://repository.apache.org/content/repositories/orgapachemahout-1060/org/apache/mahout/<https://slack-redir.net/link?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachemahout-1060%2Forg%2Fapache%2Fmahout%2F=3>

--andy

From: Andrew Palumbo 
Sent: Tuesday, January 14, 2020 4:18 PM
To: Andrew Musselman ; dev@mahout.apache.org 

Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

This is a call for volunteers as well.  Anyone may test the Release candidate;  
the more eyes the better.

Changed subject Line.

--Andy


From: Andrew Musselman 
Sent: Tuesday, January 14, 2020 1:43 PM
To: dev@mahout.apache.org 
Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for testing and vote

I will have time Thursday afternoon; thanks Andy

On Tue, Jan 14, 2020 at 05:52 Trevor Grant  wrote:

> I can test tomorrow.
>
> On Tue, Jan 14, 2020 at 1:41 AM Andrew Palumbo 
> wrote:
>
> > Hey all,
> > I've uploaded RC3 to nexus for scala 2.11 and scala 2.12.  I already have
> > seen a small issue while building the RC.  All artifacts are available in
> > staging:
> >
> >
> >
> https://repository.apache.org/service/local/repositories/orgapachemahout-1060/content/org/apache/mahout/
> >
> > We should test this release well, as it is a heavy refactor, We still
> have
> > not tested /bin/mahout.
> >
> > So we need three binding +1s for this release.
> >
> > For testing, lease do the following:
> >  1. Download build and test the src archive.
> >  2. Run a job CCO e.g which uses the CLI.
> >  3. run on a cluster/pseudo or otherwise.
> >  4. Verify Sigs and hashes.
> >
> > I noticed that many people are busy this week.  I am wondering if we
> > can/should leave the Vote open longer than 72hrs.
> >
> > Or if we can work out all trivial issues, like naming and this extra
> > source distribution, and I'll build RC4.
> >
> >
> > Thank you,
> >
> > Andy
> >
> >
>

Re: [VOTE] MAHOUT-14.1 RC3 - call for [Volunteers] and vote

[Andrew Palumbo]

This is a call for volunteers as well.  Anyone may test the Release candidate;  
the more eyes the better.

Changed subject Line.

--Andy


From: Andrew Musselman 
Sent: Tuesday, January 14, 2020 1:43 PM
To: dev@mahout.apache.org 
Subject: Re: [VOTE] MAHOUT-14.1 RC3 - call for testing and vote

I will have time Thursday afternoon; thanks Andy

On Tue, Jan 14, 2020 at 05:52 Trevor Grant  wrote:

> I can test tomorrow.
>
> On Tue, Jan 14, 2020 at 1:41 AM Andrew Palumbo 
> wrote:
>
> > Hey all,
> > I've uploaded RC3 to nexus for scala 2.11 and scala 2.12.  I already have
> > seen a small issue while building the RC.  All artifacts are available in
> > staging:
> >
> >
> >
> https://repository.apache.org/service/local/repositories/orgapachemahout-1060/content/org/apache/mahout/
> >
> > We should test this release well, as it is a heavy refactor, We still
> have
> > not tested /bin/mahout.
> >
> > So we need three binding +1s for this release.
> >
> > For testing, lease do the following:
> >  1. Download build and test the src archive.
> >  2. Run a job CCO e.g which uses the CLI.
> >  3. run on a cluster/pseudo or otherwise.
> >  4. Verify Sigs and hashes.
> >
> > I noticed that many people are busy this week.  I am wondering if we
> > can/should leave the Vote open longer than 72hrs.
> >
> > Or if we can work out all trivial issues, like naming and this extra
> > source distribution, and I'll build RC4.
> >
> >
> > Thank you,
> >
> > Andy
> >
> >
>

[VOTE] MAHOUT-14.1 RC3 - call for testing and vote

[Andrew Palumbo]

Hey all,
I've uploaded RC3 to nexus for scala 2.11 and scala 2.12.  I already have seen 
a small issue while building the RC.  All artifacts are available in staging:

https://repository.apache.org/service/local/repositories/orgapachemahout-1060/content/org/apache/mahout/

We should test this release well, as it is a heavy refactor, We still have not 
tested /bin/mahout.

So we need three binding +1s for this release.

For testing, lease do the following:
 1. Download build and test the src archive.
 2. Run a job CCO e.g which uses the CLI.
 3. run on a cluster/pseudo or otherwise.
 4. Verify Sigs and hashes.

I noticed that many people are busy this week.  I am wondering if we can/should 
leave the Vote open longer than 72hrs.

Or if we can work out all trivial issues, like naming and this extra source 
distribution, and I'll build RC4.


Thank you,

Andy

Planning for next Hangout

[Andrew Palumbo]

Hey All, I'm Trying to find a good day for the next hangou/meeting:

https://docs.google.com/document/d/10oWUL890PsFc8Bhbnd10OxLN7ef9wGcEYhpg9-L1Fnc/edit?usp=sharing



  1.  Cancelled last Friday, January 11 2020 due to several conflicts and 
illness.

  2.  This coming Friday is a no go for @rawkintrevo, on vacation.

  3.  The following Friday, January 24, 2020?

The link above has comment permissions for all please let me know if you'd like 
to be added to edit.

Could all that are interested comment on the above link please with 
availability?  I think that there is much to discuss.

--Andy


Below please find a link to the last meeting's notes:

https://docs.google.com/document/d/1eN1l7y332vzmwn6ooXF-wsNPyi_momaeZu5WQh0MD34/edit

Re: 0 14 0 Release - mahout RC3

[Andrew Palumbo]

Hey @all I'm just getting over some kinda flu .. I should be able to cut a 
release in the next couple of days.

Just FYI- still on it.  Possibly tomorrow.

--andy

On Dec 22, 2019 11:19 PM, "Andy Palumbo - [mahout] [slack]" 
<0-14-0-rele...@mahout.mailclark.ai> wrote:

Hey guys i was trying but the holiday season got started early this year.  Fam 
in town etc.. I may not be able to cut rv3 until after Xmas. I don't figure any 
idy wants to be testing right now anyi.   Just fyi.

Happy holidays all!

I'll prob be back online btw. Xmas and new year.

Re: 0 14 0 Release - mahout RC3

[Andrew Palumbo]

Hey @all I'm just getting over some kinda flu .. I should be able to cut a 
release in the next couple of days.    

Just FYI- still on it.  Possibly tomorrow.    

--andy

Re: 0 14 0 Release - mahout

[Andrew Palumbo]

hey all - FYI - I've been under the weather and have been  unable to get an RC 
cut on the timeline that I'd hoped. I'll try to get it out in the next day or 
so, and will wait to send the vote untill sometime after new years.   

 or LMK [ @Andrew Musselman](mailto:a...@apache.org) if you'd rather that i 
Just keep a the vote open more than 72 hrs on it, and i'' send out a vote on as 
it is released.   

 no big deal i guess just figuring no one is interested on a new year's 
deadline fot a  mahout testing vote.  

 Happy new years all!  

 --andy  

-

Re: 0 14 0 Release - mahout

[Andrew Palumbo]

hey all - FYI - I've been under the weather and have been  unable to get an RC 
cut on the timeline that I'd hoped. I'll try to get it out in the next day or 
so, and will wait to send the vote untill sometime after new years.

or LMK @Andrew Musselman if you'd rather that i Just 
keep a the vote open more than 72 hrs on it, and i'' send out a vote on as it 
is released.

no big deal i guess just figuring no one is interested on a new year's deadline 
fot a  mahout testing vote.

Happy new years all!

--andy




From: 0-14-0-rele...@mahout.mailclark.ai <0-14-0-rele...@mahout.mailclark.ai> 
on behalf of Andy Palumbo - [mahout] [slack] 
<0-14-0-rele...@mahout.mailclark.ai>
Sent: Monday, December 23, 2019 2:19 AM
To: 0-14-0-rele...@mahout.mailclark.ai <0-14-0-rele...@mahout.mailclark.ai>
Subject: Re: 0 14 0 Release - mahout

Hey guys i was trying but the holiday season got started early this year.  Fam 
in town etc.. I may not be able to cut rv3 until after Xmas. I don't figure any 
idy wants to be testing right now anyi.   Just fyi.

 Happy holidays all!

I'll prob be back online btw. Xmas and new year.

Re: Mahout Docker Images

[Andrew Palumbo]

Yes seen that before- Would be great to have an image there.  I was wondering 
who controlled it.

--andy

From: Trevor Grant 
Sent: Monday, December 23, 2019 1:32 PM
To: Mahout Dev List ; Joe Olson 
Subject: Mahout Docker Images

Have you seen this? Infra controls it:
https://hub.docker.com/u/apache

Apache Beam is trying to move this there right now:
https://hub.docker.com/u/apachebeam

Re: [ANNOUNCE] Apache Mahout 0.14.1 RC-1

[Andrew Palumbo]

Signs and hashes were not generated for some artifacts. Cancelling this vote.  
Will send a link for RC2 soon.
--andy

From: Andrew Palumbo 
Sent: Saturday, December 14, 2019 10:36 PM
To: dev@mahout.apache.org ; u...@mahout.apache.org 
; priv...@mahout.apache.org 
; annou...@apache.org 
Subject: [ANNOUNCE] Apache Mahout 0.14.1 RC-1

Below please find the first candidate release for Mahout 14.1

https://repository.apache.org/content/repositories/orgapachemahout-1058

Off the bat I can see some trivial naming issues that we need to work out.  
This is in part due to the nested module structure.. We must come to a 
consensus on the naming there.

Otherwise please download test and verify the artifacts..


  1.  download and run tests on srv.tar.gz distribution
  2.  try running an app with some of he newly refactored code i.e. something 
that needs both `hdfs` and `core`
  3.  if possible please try to run on a cluster.  If not. a pseudocluster 
would be great.  We need some coverage on HDFS at least.
  4.  try running something with the `/bin/mahout` command, this may need an 
overhaul
  5.  verify artifacts

We need 3 (+1) Binding PMC votes.

The vote will expire in 72 hours.. Tuesday Dec 17 at 7:30.

Thank you very much.  This has been a long effort, and I believe that we can 
get the ball rolling with less mundane work, and regular releases.

--andy

[ANNOUNCE] Apache Mahout 0.14.1 RC-1

[Andrew Palumbo]

Below please find the first candidate release for Mahout 14.1

https://repository.apache.org/content/repositories/orgapachemahout-1058

Off the bat I can see some trivial naming issues that we need to work out.  
This is in part due to the nested module structure.. We must come to a 
consensus on the naming there.

Otherwise please download test and verify the artifacts..


  1.  download and run tests on srv.tar.gz distribution
  2.  try running an app with some of he newly refactored code i.e. something 
that needs both `hdfs` and `core`
  3.  if possible please try to run on a cluster.  If not. a pseudocluster 
would be great.  We need some coverage on HDFS at least.
  4.  try running something with the `/bin/mahout` command, this may need an 
overhaul
  5.  verify artifacts

We need 3 (+1) Binding PMC votes.

The vote will expire in 72 hours.. Tuesday Dec 17 at 7:30.

Thank you very much.  This has been a long effort, and I believe that we can 
get the ball rolling with less mundane work, and regular releases.

--andy

14.1 release- not from slack email :) -- that app will be the death of me

[Andrew Palumbo]

@rawkintrevo:

Did you get an answer here RE: meeting tomorrow, Sat. 12/?14?

I can meet tomorrow afternoon, the later the better.

I'd forgotten several things this week. To cut an RC there are some new changes 
that need to be made to the release master's .m2.settings.xml (me? I'm fine 
with doing it, but have instructions if someone else wants to take crack 
first)..

I'd forgotten about my anniversary so wont finish anything more tonight, and 
prob will be out of pocket till at least 2:00PM PST tomorrow. I can try to get 
the change-scala-version.sh [1] worked out and committed by then, if someone 
wants to cut an RC before tomorrow afternoon can do a find/replace if the 
following [WIP] PR is not working (still needs testing).


[1] 
https://github.com/apache/mahout/pull/385



New release commands [2]:
1. mvn clean package -Papache-release release:prepare
2. mvn -Papache-releaserelease release:perform

Instructions for new ~/.m2/settings.xml [2]:



This directory is for release related scrips and resources
Settings.xml: A setting file needed in the /.m2/ directory In order to sign 
artifacts All lines which need to be replace are marked with an  comment.http://maven.apache.org/SETTINGS/1.1.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 
http://maven.apache.org/xsd/settings-1.1.0.xsd;>
 
${user.home}/.m2/repository
 true
 false  
  
 
  apache-release
  
THE_EIGHT_DIGIT_GPG_SIGNING_KEY_NAME
PASSPHRASE_TO_YOUR_SIGNING_KEY

mahout.releases::default::https://repository.apache.org/service/local/staging/deploy/maven2/
 
https://repository.apache.org/service/local/staging/deploy/maven2/
   

   
 true
   
  
  THE_EIGHT_DIGIT_GPG_SIGNING_KEY_NAME
  
 
   
   
 
 
   apache.snapshots.https
   website
   ASF_LDAP_USERNAME   

   ASF_LDAP_PASSWORD   

 
 
   apache.website
   ASF_LDAP_USRNAME 

   ASF_LDAP_PASSWORD

   664
   775
  
 
   apache.releases.https
   ASF_LDAP_USERNAME
   ASF_LDAP_PASSWORD

   ${user.home}/.ssh/id_rsa
   PASSWORD_TO_YOUR_PRIVATE_SSH_KEY 

   664
   775
   
  
  THE_EIGHT_DIGIT_GPG_SIGNING_KEY_NAME
  PASSPHRASE_TO_YOUR_SIGNING_KEY
 
 
   apache.website 
   ASF_LDAP_USERNAME   

   664
   775
 
   
 Required settings for a release via maven 3.3.9 
are:ASF_LDAP_USERNAME
ASF_LDAP_PASSWORDin order to create an scp ssession with the repository to 
upload artifact in the deploy phase.THE_EIGHT_DIGIT_GPG_SIGNING_KEY_NAME
PASSPHRASE_TO_YOUR_SIGNING_KEYAre used to sign and verify commits. Theese are 
required variables which are used when creating a mahout release (and only if 
you inted to release.). The keyname: THE_EIGHT_DIGIT_GPG_SIGNING_KEY_NAME for 
lack of any imagination listed here must be recognized by ASF, and listed in 
the mahout /Distribution/KEYS file.Edit these variables to deploy to an other 
URL: 
mahout.releases::default::https://repository.apache.org/service/local/staging/deploy/maven2/

https://repository.apache.org/service/local/staging/deploy/maven2/

[2] 
https://github.com/apache/mahout/blob/8794c42f910ed6fc3ac93d4081a88385520cc84b/release/README.md

Fyi, there is a kown bug with mvn (i dont have it handy and have to run) 
occassionally, between release:prepare and release:perform the version number 
will be set to the next iteration without the deploy goal running..

in this case one must reset all the version back to 14.1 and run mvn deploy (or 
possibly mvn release:perform? .. itsbeen a while, but point is berfore running 
release:perform, sometimes need to rolback by hand and release otherwise you 
end up releasing 14.2-SNAPSHOT

I hope to be online by around 2:PM tomorrow. thanks

--andy

[MEETING NOTES] 10 AM Friday 6 Jan 2019. Google Hangouts

[Andrew Palumbo]

Mahout meeting notes  12.6.2019

==



A meeting was held today, Friday 6 Dec 2019 to discuss to discuss the current 
state of the project, planned releases and a general path forward.

Joe Olson, Andrew Palumbo and Trevor Grant met via Google Hangouts at 10:15 AM.


Early discussion was based around AP and TG’s loose and quickly put together 
agenda and ideas. AP started the unofficial agenda doc <10 mins before the 
meeting start, so the agenda was quick n dirty.


An agreement was made early on by TG and AP to focus on the release as the 
build is currently working, and releases are deploying artifacts for Scala 
2.11, scala 2.12, pegged to Java 1.8 and mvn 3.3.9.  A heavy refactoring effort 
was made to After fixing the build, by revamping some very old poms and 
reverting back to the parertnt `Apache pom.xml` `release` goal and adding some 
new information to the release master’s  `.m2/setings.xml`, we are able to 
release and deploy artifacts with Java 1.8, cross compiled for Scala 2.11 and 
Scala 2.12.


https://repository.apache.org/#nexus-search;gav~org.apache.mahout~mahout-core_2.12kw,versionexpand

https://repository.apache.org/#nexus-search;gav~org.apache.mahout~mahout-hdfs_2.11kw,versionexpand


A release board was created:


https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=348


And some minor issues were added.


A decision was made to move Docker files and certain planned AWS infrastructure 
as code (TerraForm) slated for the 14.1 release off the central repository, and 
onto both dockerhub.io under a newly created mahout namespace, JO will be 
handling the task of creating the hub.docker.com/<https://hub.docker.com/> 
"Mahout" organization, and moving the Docker files to that space.  
[MAHOUT-2074<https://issues.apache.org/jira/browse/MAHOUT-2074>]



AP will be creating a mahout-contrib repo on his personal page to be merged in 
later with some terraform code and examples, etc probably borrowing heavily 
from Pulsar and spark: https://github.com/apache/pulsar/tree/master/deployment. 
As well AP will (has) begin/begun leveraging some off project time onto this 
mahout-contrib package, or at least is keeping much in the org.apache.mahout 
namespace.  Some work has already been done with NiFi and MiNifi for an SDR 
project under the org.apache.mahout namespace which will be available in the 
mahout-contrib package or an other stand alone package [NOT DISCUSSED] at 
meeting. Forgot to bring up.


AP will fix the `change-scala-version.sh` script, [MAHOUT-2080] and will bump 
the scala version in master over the weekend 
[MAHOUT-20<https://issues.apache.org/jira/browse/MAHOUT-2074>82], at which 
point we will call a code freeze.  And attempt to release by next weekend 
(after cutting an RC).



TG was able to get Jenkins running again, building snapshots, fixing 
[MAHOUT-2073<https://issues.apache.org/jira/browse/MAHOUT-2073>].


JO will look into some other projects build-chains 
MAHOUT-2076<https://issues.apache.org/jira/browse/MAHOUT-2076>, and consider a 
scripts to cut down on RC creation and Release Deployment time by having a 
single script with all release commands, similar to Apache Spark (Pulsar was 
discussed as a reference but the project is moving quickly and they’ve 
refactored their build since last I’d (AP) looked, in fact, deploying pulsar is 
as simple as `mvn clean deploy`.


https://github.com/apache/pulsar/blob/master/.test-infra/jenkins/job_pulsar_release_nightly_snapshot.groovy.


Spark and Flink should have good examples…. E.g:

Spark:

https://github.com/apache/spark/blob/master/dev/make-distribution.sh

https://github.com/apache/spark/tree/master/dev/create-release

Flink:

https://github.com/apache/flink/tree/master/tools/releasing



TG will work on zeppelin integration for some easy mahout-python-ggplot2 
examples.


There was discussion of using the The US Census Api for a data examples.


A long running issue was resolved 
[MAHOUT-2023<https://issues.apache.org/jira/browse/MAHOUT-2023>] Broken Scopt 
Classes:


*ISSUE*:  we have no way of testing this, we need @pat to take a look.  With 
the nightly snapshots being build, the current version in master is available 
in NEXUS:


[JENKINS] Archiving 
/home/jenkins/jenkins-slave/workspace/mahout-nightly/community/spark-cli-drivers/target/mahout-spark-cli-drivers_2.11-14.1-SNAPSHOT.jar
 to 
org.apache.mahout/mahout-spark-cli-drivers_2.11/14.1-20191206.193308-1/mahout-spark-cli-drivers_2.11-14.1-20191206.193308-1.jar



We spoke quickly today, and these notes were compiled to the best of my 
recollection.  If I missed anything, please bring let me know.





Trevor’s Agenda:

  1.  Release # Addressed

 *   Path to release # addressed

 *   Steps -> jira tickets # addressed

 *   Code freeze date # addressed Monday, 9 Dec 2019.

  2.  Other Misc..#  discussed and addressed



Andy’s agenda..


# RELE

[jira] [Created] (MAHOUT-2082) bump master to 2.12

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2082:
--

 Summary: bump master to 2.12
 Key: MAHOUT-2082
 URL: https://issues.apache.org/jira/browse/MAHOUT-2082
 Project: Mahout
  Issue Type: Task
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
 Fix For: 14.1






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MAHOUT-2080) fix change-scala-version.sh

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2080:
--

 Summary: fix change-scala-version.sh
 Key: MAHOUT-2080
 URL: https://issues.apache.org/jira/browse/MAHOUT-2080
 Project: Mahout
  Issue Type: Bug
Reporter: Andrew Palumbo






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MAHOUT-2079) investigat Cuda as a backend for in core matricies.

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2079:
--

 Summary: investigat Cuda as a backend for in core matricies.
 Key: MAHOUT-2079
 URL: https://issues.apache.org/jira/browse/MAHOUT-2079
 Project: Mahout
  Issue Type: Wish
Affects Versions: 0.14.1
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
 Fix For: 14.2


with nvidia card  memories now exceeding what used to be max for some 
machines.. 16 G easily it would be good to be able to dumb into Cuda memory 
directly from a statment or assignment.  look into backing in-core matrices 
with CIDA memory or CUDA Shared memory.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[proposal] look into Apache Arrow as one in a series of incore matrix back ends.

[Andrew Palumbo]

it would bee good to explore different memory structures as backs to mahout 
in-core matricies.. e.g apache arrow NVDIA CUDA..etc.  it would be good to 
start looking at these... not for this release.. IMO. starting tickets..

[jira] [Created] (MAHOUT-2078) investigat Apache arrow as a backend for in core matricies.

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2078:
--

 Summary: investigat Apache arrow as a backend for in core 
matricies.
 Key: MAHOUT-2078
 URL: https://issues.apache.org/jira/browse/MAHOUT-2078
 Project: Mahout
  Issue Type: Wish
Affects Versions: 0.14.0
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
 Fix For: 14.1


Start lokinging into Apache Arrow as an alterneate memory back end for mahout 
in-core and matrices.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: ~/m2 settings to release mahout

[Andrew Palumbo]

OOPS... sorr. I sent a bad file in the previous email.  this is the correct 
one.. I'm atepmting to make a PR out of it now, but somethings going on with my 
mac PGP that is distracting me...

Sorry asbou that Andrew.. Hope It was too late to have wasted any of your time. 
soo...

theReal~/.ms/Settings.xml:
```

http://maven.apache.org/SETTINGS/1.1.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 
http://maven.apache.org/xsd/settings-1.1.0.xsd;>


  ${user.home}/.m2/repository
  true
  false

   
   
  
 apache-release
 
   THE_EIGHT_DIGIT_GPG_SIGNING_KEY_YOURE_USING
   PASSPHRASE_TO_YOUR_SIGNING_KEY
   
mahout.releases::default::https://repository.apache.org/service/local/staging/deploy/maven2/
  
https://repository.apache.org/service/local/staging/deploy/maven2/

   

   

  true

   
   
THE_EIGHT_DIGIT_GPG_SIGNING_KEY_YOURE_USING
   
  


  
  
apache.snapshots.https
website
ASF_LDAP_USERNAME  
  # EDIT this
ASF_LDAP_PASSWORD  # EDIT this
 

  
  
apache.website
ASF_LDAP_USRNAME   
# EDIT this
ASF_LDAP_PASSWORD   # 
EDIT this
664
775
  

  
  
apache.releases.https
ASF_LDAP_USERNAME
ASF_LDAP_PASSWORD  
# EDIT this
${user.home}/.ssh/id_rsa
PASSWORD_TO_YOUR_PRIVATE_SSH_KEY   
# EDIT this
664
775

  

  
 THE_EIGHT_DIGIT_GPG_SIGNING_KEY_YOURE_USING
 PASSPHRASE_TO_YOUR_SIGNING_KEY
   # EDIT this
  

  
  
apache.website/id> 
ASF_LDAP_USERNAME  
  # EDIT this
664
775
  




________
From: Andrew Palumbo 
Sent: Friday, November 22, 2019 1:03 AM
To: dev@mahout.apache.org 
Subject: ~/m2 settings to release mahout

These are the current .m2/settings setting the I have successfully used to 
stage a mahout release candidate.

Requirements:
Java 1.8
Maven >=3.3.9

Note the upgrade of A http://maven.apache.org/xsd/settings-1.1.0.xsd to 1.1.10 
from  1.0.o which is a pretty major upgrade in terms of the elements used.



http://maven.apache.org/SETTINGS/1.1.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 
http://maven.apache.org/xsd/settings-1.1.0.xsd;>


  ${user.home}/.m2/repository
  true
  false

   
   
  
 apache-release
 
 
THE_EIGHT_DIGIT_GPG_SIGNING_KEY_YOURE_USING
 PASSPHRASE_TO_YOUR_SIGNING_KEY
 
mahout.releases::default::https://repository.apache.org/service/local/staging/deploy/maven2/

https://repository.apache.org/service/local/staging/deploy/maven2/
 
   
   

  true


 ASF_LDAP_USRNAME   
# EDIT this

  


  
  
apache.snapshots.https
website
ASF_LDAP_USRNAME   
 # EDIT this
 ASF_LDAP_USRNAME  # EDIT this
 

  
  
apache.website
ASF_LDAP_USRNAME   
# EDIT this
MY_ASF_LDAP_PASSWORD   # 
EDIT this
664
775
  

  
  
apache.releases.https
ASF_LDAP_USERNAME
 ASF_LDAP_PASSWORD 
 # EDIT this
 ${user.home}/.ssh/id_rsa
PASSWORD_TO_YOUR_PRIVATE_SSH_KEY   
# EDIT this
664
775

  

  
 THE_EIGHT_DIGIT_GPG_SIGNING_KEY_YOURE_USING
 PASSPHRASE_TO_YOUR_SIGNING_KEY
   # EDIT this
  

  
  
apache.website/id> 
ASF_LDAP_USERNAME  
  # EDIT this
664
775
  

[jira] [Created] (MAHOUT-2077) Clean up aftor behemoth PR

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2077:
--

 Summary: Clean up aftor behemoth PR 
 Key: MAHOUT-2077
 URL: https://issues.apache.org/jira/browse/MAHOUT-2077
 Project: Mahout
  Issue Type: New Feature
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo


left some files around afrter pr#382 -  MAHOUT-2071.  there are sume dupliucate 
files and extra files around.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MAHOUT-2076) add a /release directory with a skeleton settings.xml for releases

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2076:
--

 Summary: add a /release directory with a skeleton settings.xml for 
releases
 Key: MAHOUT-2076
 URL: https://issues.apache.org/jira/browse/MAHOUT-2076
 Project: Mahout
  Issue Type: New Feature
  Components: release
Affects Versions: 0.14.0
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
 Fix For: 14.1


add a  {{/release}} diectory with {{settins.xm}}l and all necessary variable 
changes to make a release with {{mvn release:prepare -Papache-release}} and



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

~/m2 settings to release mahout

[Andrew Palumbo]

These are the current .m2/settings setting the I have successfully used to 
stage a mahout release candidate.

Requirements:
Java 1.8
Maven >=3.3.9

Note the upgrade of A http://maven.apache.org/xsd/settings-1.1.0.xsd to 1.1.10 
from  1.0.o which is a pretty major upgrade in terms of the elements used.



http://maven.apache.org/SETTINGS/1.1.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 
http://maven.apache.org/xsd/settings-1.1.0.xsd;>


  ${user.home}/.m2/repository
  true
  false

   
   
  
 apache-release
 
 
THE_EIGHT_DIGIT_GPG_SIGNING_KEY_YOURE_USING
 PASSPHRASE_TO_YOUR_SIGNING_KEY
 
mahout.releases::default::https://repository.apache.org/service/local/staging/deploy/maven2/

https://repository.apache.org/service/local/staging/deploy/maven2/
 
   
   

  true


 ASF_LDAP_USRNAME   
# EDIT this

  


  
  
apache.snapshots.https
website
ASF_LDAP_USRNAME   
 # EDIT this
 ASF_LDAP_USRNAME  # EDIT this
 

  
  
apache.website
ASF_LDAP_USRNAME   
# EDIT this
MY_ASF_LDAP_PASSWORD   # 
EDIT this
664
775
  

  
  
apache.releases.https
ASF_LDAP_USERNAME
 ASF_LDAP_PASSWORD 
 # EDIT this
 ${user.home}/.ssh/id_rsa
PASSWORD_TO_YOUR_PRIVATE_SSH_KEY   
# EDIT this
664
775

  

  
 THE_EIGHT_DIGIT_GPG_SIGNING_KEY_YOURE_USING
 PASSPHRASE_TO_YOUR_SIGNING_KEY
   # EDIT this
  

  
  
apache.website/id> 
ASF_LDAP_USERNAME  
  # EDIT this
664
775
  

Re: 14.1 release

[Andrew Palumbo]

+1


From: Trevor Grant 
Sent: Wednesday, October 16, 2019 1:12 PM
To: Mahout Dev List 
Subject: Re: 14.1 release

I mean, that was the "build breaker" right?

Whatever you did by hand, maybe we make a script and call it 14.1?


On Mon, Oct 14, 2019 at 9:34 PM Andrew Palumbo  wrote:

> Btw I just mentioned on another thread binaries for the 14.1-SNAPSHOT are
> available:
>
> The default is spark 2.4.4 and scala_2.12:
>
>
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/mahout/mahout-spark_2.12/14.1-SNAPSHOT/
>
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/mahout/mahout-core_2.12/14.1-SNAPSHOT/
>
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/mahout/mahout-hdfs_2.12/14.1-SNAPSHOT/
>
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/mahout/mahout-spark-cli-drivers_2.12/14.1-SNAPSHOT/
>
> Those are pushed by hand using `mvn deploy`
>
> I've been a bit under the weather since I pushed those, but I think that
> the release shouldn't. Be too far off.  I hopefully will be able to give it
> a look Wednesday  or Sunday.
>
>
> On Oct 14, 2019 7:07 PM, Andrew Palumbo  wrote:
>
>
> >> I'll OHSU it to a mahout-14.1
>
>
> Should read I'll *push* it to a mahout-14.1 branch.
>
> I'm laid up today.  Hopefully tomorrow.
>
>
>
>
> On Oct 9, 2019 3:44 PM, Andrew Palumbo  wrote:
>
> Anybody have a status on the 14.1 release?  I noticed that it was pushed
> to a mahout-14.2 branch. Bit sure if the snapshots are set at 14.2 but this
> release should be 14.1 correct?
>
> Assuming 72 hr lazy consensus I'll OHSU it to a mahout-14.1 branch and
> delete mahout-14.2
>
> Andy
>
>
>

Community/planning call. 11/15

[Andrew Palumbo]

Hey all,


Checking in to see if we were doing a call today?

Is the usual time 10:00pst?

Andy

Proposal for a New Serialization, De-serialization and Memory Scheme for [Spark] distribution of Mahout AbstractVectors and Mahout Abstract Matrices: Direct to Main (off heap), JVM (on heap), GPU and

[Andrew Palumbo (via Google Docs)]

I've shared an item with you:

Proposal for a New Serialization, De-serialization and  Memory Scheme for  
[Spark] distribution of Mahout AbstractVectors and Mahout Abstract  
Matrices: Direct to Main (off heap), JVM (on heap), GPU and other devices

https://docs.google.com/document/d/18RybVEpjqjDU_cCzwM6dtS3ZZkd1tDvCYlkhqzloS-4/edit?usp=sharing=5dbf8960

It's not an attachment -- it's stored online. To open this item, just click  
the link above.

[jira] [Created] (MAHOUT-2075) Cross Compile and Deploy artifacts for Scala 2.11 and 2.12

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2075:
--

 Summary: Cross Compile and Deploy artifacts for Scala 2.11 and 2.12
 Key: MAHOUT-2075
 URL: https://issues.apache.org/jira/browse/MAHOUT-2075
 Project: Mahout
  Issue Type: New Feature
Affects Versions: 0.14.0
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
 Fix For: 14.1


[~pferrel], Mahout's current largest known customer, has need for Scala 2.11.  
We Should be easily able to If only by hand for this version, compile and 
deploy Scala 2.11 modules.  For the next releases, we may want to consider 
keeping a Scala 2.11 branch.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Build failed in Jenkins: mahout-nightly #2299

[Andrew Palumbo]

to clarify and fix a statement:

>>We do have staged binary artifacts, currently for the 14.1 release pegged to 
>>scala 2.12, spark 2.4.4,  java 8 and maven 3.3.9+.  We had  touched on, in 
>>slack, the idea of a realease of these binaries,  If I/we  can't get this 
>>assembly together in a reasonable amount of time

I'm actually not advocating for the above, just bringing it up here.  I'd 
actually like to get the distros working.

 ...but then I'd *_also like_* to get the Dockerfiles polished, and I'd 
*_like_* to get some Terraform code in /examples to launch on AWS, and I'd 
*_like_* to move move some Real-time synthetic data running through the 
incubating Apache Data Sketches Project into DRMs to compare their streaming 
SVD  approximation with our DSSVD's.

I'd really like to bake a Zepplin/Mahout notebook server AMI with Apache [NifI, 
Flink, Spark, DataSketches] Mahout examples...

could keep going, but we need to get a release

IMO it would be worth getting the binary and source Mahout distributions 
working, so that we could at least easily show some examples. on the webpage.

Thanks,

--andy



--andy

____________
From: Andrew Palumbo 
Sent: Tuesday, October 22, 2019 4:09 PM
To: Mahout Dev List ; Dmitriy Lyubimov 
; Trevor Grant ; Andrew 
Musselman ; Pat Ferrel ; Isabel 
Drost 
Subject: Re: Build failed in Jenkins: mahout-nightly #2299

Trevor.  Yeah- I did.  thanks to microsoft.


I filed a ticket w INFRA, Gavin McDonald, about it. we must enforce mvn 3.3.9.. 
.

https://issues.apache.org/jira/browse/INFRA-19292

This email is from last week when i was working on this Thursday night/Friday 
morn.

Have People been getting emails since?

Also What do you guys think- should we change the email ti Issues@?  rather 
than dev@?

There is also a builds@ which I was unaware of.. I (we, some of the PMC) should 
to subscribe to it, as Gavin Pointed out on the ticket.

I'm, going to do some mahout work today.  Pretty sure I see the release error.

also BTW. when I run. PRs from the refactored Branch (mahout-14.1), It is in 
the green... Maybe time I write up a formal PR to Merge that branch, I think 
that we should be able to release soon..  (I'm not sure that there is a reason 
not to have this on master at this point.

Was looking at the assembly last night again, and I have to do a rollback, and 
test of what i think may be off in the bin.xml, and root pom.


We do have staged binary artifacts, currently for the 14.1 release pegged to 
scala 2.12, spark 2.4.4,  java 8 and maven 3.3.9+.  We had  touched on, in 
slack, the idea of a realease of these binaries,  If I/we  can't get this 
assembly together in a reasonable abount of time



@rawkintrevo, @akm, @pferrel, @Dmitriy Lyubimov<mailto:dlyubi...@apache.org> 
@Isabel Drost<mailto:isa...@apache.org>, @Mahout Dev 
List<mailto:dev@mahout.apache.org>  thoughts?

Thanks,


Andy

From: Trevor Grant 
Sent: Thursday, October 17, 2019 7:07 AM
To: Mahout Dev List 
Subject: Re: Build failed in Jenkins: mahout-nightly #2299

Guessing Palumbo turned this back on?

Maybe set them to not send messages to dev until we get the Jenkins build
working right again.


On Thu, Oct 17, 2019 at 5:38 AM Apache Jenkins Server <
jenk...@builds.apache.org> wrote:

> See <https://builds.apache.org/job/mahout-nightly/2299/display/redirect>
>
> Changes:
>
>
> --
> Started by user apalumbo
> Running as SYSTEM
> [EnvInject] - Loading node environment variables.
> Building remotely on H31 (ubuntu) in workspace <
> https://builds.apache.org/job/mahout-nightly/ws/>
> ERROR: A Maven installation needs to be available for this project to be
> built.Either your server has no Maven installations defined, or the
> requested Maven version does not exist.
> Retrying after 10 seconds
> ERROR: A Maven installation needs to be available for this project to be
> built.Either your server has no Maven installations defined, or the
> requested Maven version does not exist.
> Retrying after 10 seconds
> ERROR: A Maven installation needs to be available for this project to be
> built.Either your server has no Maven installations defined, or the
> requested Maven version does not exist.
>

Re: Build failed in Jenkins: mahout-nightly #2299

[Andrew Palumbo]

Thanks, Tg.

From: Trevor Grant 
Sent: Tuesday, October 22, 2019 4:30 PM
To: Andrew Palumbo 
Cc: Mahout Dev List ; Dmitriy Lyubimov 
; Andrew Musselman ; Pat 
Ferrel ; Isabel Drost 
Subject: Re: Build failed in Jenkins: mahout-nightly #2299

I never got any additional build error notices.  I'm +1 to utilizing derelict 
email addresses.

On Tue, Oct 22, 2019 at 3:18 PM Andrew Palumbo 
mailto:ap@outlook.com>> wrote:
I'm actually not advocating for this, just bringing it up here.  I'd actually 
like to get the distros working.  But then Id like to get the Dockerfiles 
polished, and I'd like to get Some Terraform code in /examples to launch on 
aws, and I'd like to move move some Real-time synthetic data runnining through 
the incubating  DataSketches project into Drms to compare their SVD  
approximation with our DSSVD's.  Putting together a zepplin/apache 
everything/mahout AMI for examples...

could keep going..

IMO it would be worth getting the distros working, so that we could at least 
have examples running.

>>We do have staged binary artifacts, currently for the 14.1 release pegged to 
>>scala 2.12, spark 2.4.4,  java 8 and maven 3.3.9+.  We had  touched on, in 
>>slack, the idea of a realease of these binaries,  If I/we  can't get this 
>>assembly together in a reasonable amount of time


--andy

________
From: Andrew Palumbo mailto:ap@outlook.com>>
Sent: Tuesday, October 22, 2019 4:09 PM
To: Mahout Dev List mailto:dev@mahout.apache.org>>; 
Dmitriy Lyubimov mailto:dlyubi...@apache.org>>; Trevor 
Grant mailto:trevor.d.gr...@gmail.com>>; Andrew 
Musselman mailto:andrew.mussel...@gmail.com>>; Pat 
Ferrel mailto:p...@actionml.com>>; Isabel Drost 
mailto:isa...@apache.org>>
Subject: Re: Build failed in Jenkins: mahout-nightly #2299

Trevor.  Yeah- I did.  thanks to microsoft.


I filed a ticket w INFRA, Gavin McDonald, about it. we must enforce mvn 3.3.9.. 
.

https://issues.apache.org/jira/browse/INFRA-19292

This email is from last week when i was working on this Thursday night/Friday 
morn.

Have People been getting emails since?

Also What do you guys think- should we change the email ti Issues@?  rather 
than dev@?

There is also a builds@ which I was unaware of.. I (we, some of the PMC) should 
to subscribe to it, as Gavin Pointed out on the ticket.

I'm, going to do some mahout work today.  Pretty sure I see the release error.

also BTW. when I run. PRs from the refactored Branch (mahout-14.1), It is in 
the green... Maybe time I write up a formal PR to Merge that branch, I think 
that we should be able to release soon..  (I'm not sure that there is a reason 
not to have this on master at this point.

Was looking at the assembly last night again, and I have to do a rollback, and 
test of what i think may be off in the bin.xml, and root pom.


We do have staged binary artifacts, currently for the 14.1 release pegged to 
scala 2.12, spark 2.4.4,  java 8 and maven 3.3.9+.  We had  touched on, in 
slack, the idea of a realease of these binaries,  If I/we  can't get this 
assembly together in a reasonable abount of time



@rawkintrevo, @akm, @pferrel, @Dmitriy Lyubimov<mailto:dlyubi...@apache.org> 
@Isabel Drost<mailto:isa...@apache.org>, @Mahout Dev 
List<mailto:dev@mahout.apache.org>  thoughts?

Thanks,


Andy

From: Trevor Grant mailto:trevor.d.gr...@gmail.com>>
Sent: Thursday, October 17, 2019 7:07 AM
To: Mahout Dev List mailto:dev@mahout.apache.org>>
Subject: Re: Build failed in Jenkins: mahout-nightly #2299

Guessing Palumbo turned this back on?

Maybe set them to not send messages to dev until we get the Jenkins build
working right again.


On Thu, Oct 17, 2019 at 5:38 AM Apache Jenkins Server <
jenk...@builds.apache.org<mailto:jenk...@builds.apache.org>> wrote:

> See <https://builds.apache.org/job/mahout-nightly/2299/display/redirect>
>
> Changes:
>
>
> --
> Started by user apalumbo
> Running as SYSTEM
> [EnvInject] - Loading node environment variables.
> Building remotely on H31 (ubuntu) in workspace <
> https://builds.apache.org/job/mahout-nightly/ws/>
> ERROR: A Maven installation needs to be available for this project to be
> built.Either your server has no Maven installations defined, or the
> requested Maven version does not exist.
> Retrying after 10 seconds
> ERROR: A Maven installation needs to be available for this project to be
> built.Either your server has no Maven installations defined, or the
> requested Maven version does not exist.
> Retrying after 10 seconds
> ERROR: A Maven installation needs to be available for this project to be
> built.Either your server has no Maven installations defined, or the
> requested Maven version does not exist.
>

Re: Build failed in Jenkins: mahout-nightly #2299

[Andrew Palumbo]

I'm actually not advocating for this, just bringing it up here.  I'd actually 
like to get the distros working.  But then Id like to get the Dockerfiles 
polished, and I'd like to get Some Terraform code in /examples to launch on 
aws, and I'd like to move move some Real-time synthetic data runnining through 
the incubating  DataSketches project into Drms to compare their SVD  
approximation with our DSSVD's.  Putting together a zepplin/apache 
everything/mahout AMI for examples...

could keep going..

IMO it would be worth getting the distros working, so that we could at least 
have examples running.

>>We do have staged binary artifacts, currently for the 14.1 release pegged to 
>>scala 2.12, spark 2.4.4,  java 8 and maven 3.3.9+.  We had  touched on, in 
>>slack, the idea of a realease of these binaries,  If I/we  can't get this 
>>assembly together in a reasonable amount of time


--andy

____________
From: Andrew Palumbo 
Sent: Tuesday, October 22, 2019 4:09 PM
To: Mahout Dev List ; Dmitriy Lyubimov 
; Trevor Grant ; Andrew 
Musselman ; Pat Ferrel ; Isabel 
Drost 
Subject: Re: Build failed in Jenkins: mahout-nightly #2299

Trevor.  Yeah- I did.  thanks to microsoft.


I filed a ticket w INFRA, Gavin McDonald, about it. we must enforce mvn 3.3.9.. 
.

https://issues.apache.org/jira/browse/INFRA-19292

This email is from last week when i was working on this Thursday night/Friday 
morn.

Have People been getting emails since?

Also What do you guys think- should we change the email ti Issues@?  rather 
than dev@?

There is also a builds@ which I was unaware of.. I (we, some of the PMC) should 
to subscribe to it, as Gavin Pointed out on the ticket.

I'm, going to do some mahout work today.  Pretty sure I see the release error.

also BTW. when I run. PRs from the refactored Branch (mahout-14.1), It is in 
the green... Maybe time I write up a formal PR to Merge that branch, I think 
that we should be able to release soon..  (I'm not sure that there is a reason 
not to have this on master at this point.

Was looking at the assembly last night again, and I have to do a rollback, and 
test of what i think may be off in the bin.xml, and root pom.


We do have staged binary artifacts, currently for the 14.1 release pegged to 
scala 2.12, spark 2.4.4,  java 8 and maven 3.3.9+.  We had  touched on, in 
slack, the idea of a realease of these binaries,  If I/we  can't get this 
assembly together in a reasonable abount of time



@rawkintrevo, @akm, @pferrel, @Dmitriy Lyubimov<mailto:dlyubi...@apache.org> 
@Isabel Drost<mailto:isa...@apache.org>, @Mahout Dev 
List<mailto:dev@mahout.apache.org>  thoughts?

Thanks,


Andy

From: Trevor Grant 
Sent: Thursday, October 17, 2019 7:07 AM
To: Mahout Dev List 
Subject: Re: Build failed in Jenkins: mahout-nightly #2299

Guessing Palumbo turned this back on?

Maybe set them to not send messages to dev until we get the Jenkins build
working right again.


On Thu, Oct 17, 2019 at 5:38 AM Apache Jenkins Server <
jenk...@builds.apache.org> wrote:

> See <https://builds.apache.org/job/mahout-nightly/2299/display/redirect>
>
> Changes:
>
>
> --
> Started by user apalumbo
> Running as SYSTEM
> [EnvInject] - Loading node environment variables.
> Building remotely on H31 (ubuntu) in workspace <
> https://builds.apache.org/job/mahout-nightly/ws/>
> ERROR: A Maven installation needs to be available for this project to be
> built.Either your server has no Maven installations defined, or the
> requested Maven version does not exist.
> Retrying after 10 seconds
> ERROR: A Maven installation needs to be available for this project to be
> built.Either your server has no Maven installations defined, or the
> requested Maven version does not exist.
> Retrying after 10 seconds
> ERROR: A Maven installation needs to be available for this project to be
> built.Either your server has no Maven installations defined, or the
> requested Maven version does not exist.
>

Re: Build failed in Jenkins: mahout-nightly #2299

[Andrew Palumbo]

I don't have anymore build failures after Thursday in my inbox.  Please LMK if 
anybody is getting flooded with emails.

Thanks,

Andy

From: Andrew Palumbo 
Sent: Tuesday, October 22, 2019 4:09 PM
To: Mahout Dev List ; Dmitriy Lyubimov 
; Trevor Grant ; Andrew 
Musselman ; Pat Ferrel ; Isabel 
Drost 
Subject: Re: Build failed in Jenkins: mahout-nightly #2299

Trevor.  Yeah- I did.  thanks to microsoft.


I filed a ticket w INFRA, Gavin McDonald, about it. we must enforce mvn 3.3.9.. 
.

https://issues.apache.org/jira/browse/INFRA-19292

This email is from last week when i was working on this Thursday night/Friday 
morn.

Have People been getting emails since?

Also What do you guys think- should we change the email ti Issues@?  rather 
than dev@?

There is also a builds@ which I was unaware of.. I (we, some of the PMC) should 
to subscribe to it, as Gavin Pointed out on the ticket.

I'm, going to do some mahout work today.  Pretty sure I see the release error.

also BTW. when I run. PRs from the refactored Branch (mahout-14.1), It is in 
the green... Maybe time I write up a formal PR to Merge that branch, I think 
that we should be able to release soon..  (I'm not sure that there is a reason 
not to have this on master at this point.

Was looking at the assembly last night again, and I have to do a rollback, and 
test of what i think may be off in the bin.xml, and root pom.


We do have staged binary artifacts, currently for the 14.1 release pegged to 
scala 2.12, spark 2.4.4,  java 8 and maven 3.3.9+.  We had  touched on, in 
slack, the idea of a realease of these binaries,  If I/we  can't get this 
assembly together in a reasonable abount of time



@rawkintrevo, @akm, @pferrel, @Dmitriy Lyubimov<mailto:dlyubi...@apache.org> 
@Isabel Drost<mailto:isa...@apache.org>, @Mahout Dev 
List<mailto:dev@mahout.apache.org>  thoughts?

Thanks,


Andy

From: Trevor Grant 
Sent: Thursday, October 17, 2019 7:07 AM
To: Mahout Dev List 
Subject: Re: Build failed in Jenkins: mahout-nightly #2299

Guessing Palumbo turned this back on?

Maybe set them to not send messages to dev until we get the Jenkins build
working right again.


On Thu, Oct 17, 2019 at 5:38 AM Apache Jenkins Server <
jenk...@builds.apache.org> wrote:

> See <https://builds.apache.org/job/mahout-nightly/2299/display/redirect>
>
> Changes:
>
>
> --
> Started by user apalumbo
> Running as SYSTEM
> [EnvInject] - Loading node environment variables.
> Building remotely on H31 (ubuntu) in workspace <
> https://builds.apache.org/job/mahout-nightly/ws/>
> ERROR: A Maven installation needs to be available for this project to be
> built.Either your server has no Maven installations defined, or the
> requested Maven version does not exist.
> Retrying after 10 seconds
> ERROR: A Maven installation needs to be available for this project to be
> built.Either your server has no Maven installations defined, or the
> requested Maven version does not exist.
> Retrying after 10 seconds
> ERROR: A Maven installation needs to be available for this project to be
> built.Either your server has no Maven installations defined, or the
> requested Maven version does not exist.
>

Re: Build failed in Jenkins: mahout-nightly #2299

[Andrew Palumbo]

Trevor.  Yeah- I did.  thanks to microsoft.


I filed a ticket w INFRA, Gavin McDonald, about it. we must enforce mvn 3.3.9.. 
.

https://issues.apache.org/jira/browse/INFRA-19292

This email is from last week when i was working on this Thursday night/Friday 
morn.

Have People been getting emails since?

Also What do you guys think- should we change the email ti Issues@?  rather 
than dev@?

There is also a builds@ which I was unaware of.. I (we, some of the PMC) should 
to subscribe to it, as Gavin Pointed out on the ticket.

I'm, going to do some mahout work today.  Pretty sure I see the release error.

also BTW. when I run. PRs from the refactored Branch (mahout-14.1), It is in 
the green... Maybe time I write up a formal PR to Merge that branch, I think 
that we should be able to release soon..  (I'm not sure that there is a reason 
not to have this on master at this point.

Was looking at the assembly last night again, and I have to do a rollback, and 
test of what i think may be off in the bin.xml, and root pom.


We do have staged binary artifacts, currently for the 14.1 release pegged to 
scala 2.12, spark 2.4.4,  java 8 and maven 3.3.9+.  We had  touched on, in 
slack, the idea of a realease of these binaries,  If I/we  can't get this 
assembly together in a reasonable abount of time



@rawkintrevo, @akm, @pferrel, @Dmitriy Lyubimov 
@Isabel Drost, @Mahout Dev 
List  thoughts?

Thanks,


Andy

From: Trevor Grant 
Sent: Thursday, October 17, 2019 7:07 AM
To: Mahout Dev List 
Subject: Re: Build failed in Jenkins: mahout-nightly #2299

Guessing Palumbo turned this back on?

Maybe set them to not send messages to dev until we get the Jenkins build
working right again.


On Thu, Oct 17, 2019 at 5:38 AM Apache Jenkins Server <
jenk...@builds.apache.org> wrote:

> See 
>
> Changes:
>
>
> --
> Started by user apalumbo
> Running as SYSTEM
> [EnvInject] - Loading node environment variables.
> Building remotely on H31 (ubuntu) in workspace <
> https://builds.apache.org/job/mahout-nightly/ws/>
> ERROR: A Maven installation needs to be available for this project to be
> built.Either your server has no Maven installations defined, or the
> requested Maven version does not exist.
> Retrying after 10 seconds
> ERROR: A Maven installation needs to be available for this project to be
> built.Either your server has no Maven installations defined, or the
> requested Maven version does not exist.
> Retrying after 10 seconds
> ERROR: A Maven installation needs to be available for this project to be
> built.Either your server has no Maven installations defined, or the
> requested Maven version does not exist.
>

[jira] [Created] (MAHOUT-2074) Dockerfile(s)

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2074:
--

 Summary: Dockerfile(s) 
 Key: MAHOUT-2074
 URL: https://issues.apache.org/jira/browse/MAHOUT-2074
 Project: Mahout
  Issue Type: New Feature
Affects Versions: 14.1
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
 Fix For: 14.1


Have a [WIP] Dockerfile for which (assuming a binary release,) pulls the 
appropriate version of Spark and places both Spark and Mahout in 
\{{/opt/spark}} and {{/opt/mahout respectivly.  }}

{{Would like full build capbilities, thouigh this may be better to migrate in 
14.2 or 15.0 to an etrire new buildchain. E.g. CMake.   [I would suggest] given 
aour large amount of native code (hopefully soon to be added :))  }}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: 14.1 release

[Andrew Palumbo]

Hmm sent from slack.  Still not working ... I wrote it up and started a few 
tickets to explain.  Branch-14.1 is pretty close.



On Oct 16, 2019 10:12 AM, Trevor Grant  wrote:

I mean, that was the "build breaker" right?

Whatever you did by hand, maybe we make a script and call it 14.1?


On Mon, Oct 14, 2019 at 9:34 PM Andrew Palumbo  wrote:

> Btw I just mentioned on another thread binaries for the 14.1-SNAPSHOT are
> available:
>
> The default is spark 2.4.4 and scala_2.12:
>
>
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/mahout/mahout-spark_2.12/14.1-SNAPSHOT/
>
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/mahout/mahout-core_2.12/14.1-SNAPSHOT/
>
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/mahout/mahout-hdfs_2.12/14.1-SNAPSHOT/
>
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/mahout/mahout-spark-cli-drivers_2.12/14.1-SNAPSHOT/
>
> Those are pushed by hand using `mvn deploy`
>
> I've been a bit under the weather since I pushed those, but I think that
> the release shouldn't. Be too far off.  I hopefully will be able to give it
> a look Wednesday  or Sunday.
>
>
> On Oct 14, 2019 7:07 PM, Andrew Palumbo  wrote:
>
>
> >> I'll OHSU it to a mahout-14.1
>
>
> Should read I'll *push* it to a mahout-14.1 branch.
>
> I'm laid up today.  Hopefully tomorrow.
>
>
>
>
> On Oct 9, 2019 3:44 PM, Andrew Palumbo  wrote:
>
> Anybody have a status on the 14.1 release?  I noticed that it was pushed
> to a mahout-14.2 branch. Bit sure if the snapshots are set at 14.2 but this
> release should be 14.1 correct?
>
> Assuming 72 hr lazy consensus I'll OHSU it to a mahout-14.1 branch and
> delete mahout-14.2
>
> Andy
>
>
>

[jira] [Created] (MAHOUT-2073) Fix jenkins back up to release nightly snapshots.

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2073:
--

 Summary: Fix jenkins back up to release nightly snapshots.
 Key: MAHOUT-2073
 URL: https://issues.apache.org/jira/browse/MAHOUT-2073
 Project: Mahout
  Issue Type: Task
Affects Versions: 0.14.0
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
 Fix For: 14.1






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MAHOUT-2072) upgrade to scala 2.12 spark 2.4.4 as defaults.

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2072:
--

 Summary: upgrade to scala 2.12 spark 2.4.4 as defaults.
 Key: MAHOUT-2072
 URL: https://issues.apache.org/jira/browse/MAHOUT-2072
 Project: Mahout
  Issue Type: Task
Affects Versions: 0.14.0
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
 Fix For: 14.1






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MAHOUT-2071) fix bin.xml and assembly to allow for a binary apache-mahout-14.0.tar.gz release

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2071:
--

 Summary: fix bin.xml and assembly to allow for a binary 
apache-mahout-14.0.tar.gz release
 Key: MAHOUT-2071
 URL: https://issues.apache.org/jira/browse/MAHOUT-2071
 Project: Mahout
  Issue Type: Task
Affects Versions: 0.14.0
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
 Fix For: 14.1


the problem is theoretically solved ... we were overwriting the apache parent 
{{pom.xml}}'s[https://github.com/apache/mahout/blob/mahout-14.1/pom.xml#L21-L25|https://slack-redir.net/link?url=https%3A%2F%2Fgithub.com%2Fapache%2Fmahout%2Fblob%2Fmahout-14.1%2Fpom.xml%23L21-L25=3]
 {{apache-release}} profile in our own, ...i removed the {{apache-release}} 
profile from ours and added what was left to {{mahout-release}}.so i expect 
that it should be working now.  the thing that was breaking everything (im 
gambling this whole theory on is that by overwriting the parent 
pom'sdeclaration of {{}}:

                   ${mahout.skip.distribution}

 
[https://github.com/apache/mahout/blob/mahout-14.1/pom.xml#L512|https://slack-redir.net/link?url=https%3A%2F%2Fgithub.com%2Fapache%2Fmahout%2Fblob%2Fmahout-14.1%2Fpom.xml%23L512=3]
 
 
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (MAHOUT-2070) move releas branch for 14.1 from `branch-14.2` to `branch-14.1`

[Andrew Palumbo (Jira)]

Andrew Palumbo created MAHOUT-2070:
--

 Summary: move releas branch for 14.1 from `branch-14.2` to  
`branch-14.1`
 Key: MAHOUT-2070
 URL: https://issues.apache.org/jira/browse/MAHOUT-2070
 Project: Mahout
  Issue Type: Task
Affects Versions: 14.1
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo
 Fix For: 14.1






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: 14.1 release

[Andrew Palumbo]

Btw I just mentioned on another thread binaries for the 14.1-SNAPSHOT are 
available:

The default is spark 2.4.4 and scala_2.12:


https://repository.apache.org/content/repositories/snapshots/org/apache/mahout/mahout-spark_2.12/14.1-SNAPSHOT/

https://repository.apache.org/content/repositories/snapshots/org/apache/mahout/mahout-core_2.12/14.1-SNAPSHOT/

https://repository.apache.org/content/repositories/snapshots/org/apache/mahout/mahout-hdfs_2.12/14.1-SNAPSHOT/

https://repository.apache.org/content/repositories/snapshots/org/apache/mahout/mahout-spark-cli-drivers_2.12/14.1-SNAPSHOT/

Those are pushed by hand using `mvn deploy`

I've been a bit under the weather since I pushed those, but I think that the 
release shouldn't. Be too far off.  I hopefully will be able to give it a look 
Wednesday  or Sunday.


On Oct 14, 2019 7:07 PM, Andrew Palumbo  wrote:


>> I'll OHSU it to a mahout-14.1


Should read I'll *push* it to a mahout-14.1 branch.

I'm laid up today.  Hopefully tomorrow.




On Oct 9, 2019 3:44 PM, Andrew Palumbo  wrote:

Anybody have a status on the 14.1 release?  I noticed that it was pushed to a 
mahout-14.2 branch. Bit sure if the snapshots are set at 14.2 but this release 
should be 14.1 correct?

Assuming 72 hr lazy consensus I'll OHSU it to a mahout-14.1 branch and delete 
mahout-14.2

Andy

Re: Zeppelin status

[Andrew Palumbo]

On Oct 11, 2019 4:32 AM, Trevor Grant  wrote:

I think it _was_ merged at one point. For a long time the "Mahout Example
Notebook" lived in Zeppelin.

Yeah I thought i remembered something like that..

The issue was, just about the time we made that, Zeppelin version bumped to
Spark 2.0, which we still don't have binaries for so it was always a
question of the user having to hack their way into ti.

So the 14.1 branch does deploy binaries now the default is spark 2.4.4 and 
scala_2.12:


https://repository.apache.org/content/repositories/snapshots/org/apache/mahout/mahout-spark_2.12/14.1-SNAPSHOT/

https://repository.apache.org/content/repositories/snapshots/org/apache/mahout/mahout-core_2.12/14.1-SNAPSHOT/

https://repository.apache.org/content/repositories/snapshots/org/apache/mahout/mahout-hdfs_2.12/14.1-SNAPSHOT/

https://repository.apache.org/content/repositories/snapshots/org/apache/mahout/mahout-spark-cli-drivers_2.12/14.1-SNAPSHOT/

Those are pushed by hand using `mvn deploy`


Afaik, there's docs on how to set it up on Mahout.apache.org, which are
still accurate. You just need to make sure you compile Mahout for the
version of spark you're going to run.

Thanks..   Yeah I want to try something with Zeppelin.  I'll give it a shot 
either building or with these binaries when I can.

Also be advised, the Spark "fat" jar is _NOT_ required.

It'd also be cool to looking into hosting notebooks on mahout.apache.org,
but I don't have cycles for that atm.


 +1


tg



On Wed, Oct 9, 2019 at 5:30 PM Andrew Palumbo  wrote:

> Hey all.. I'm working on some simple Mahout integration programs.. they're
> publicly available on my GitHub.
>
> https://github.com/andrewpalumbo/mahoutTestApps
>
> (Need to push some  hanged from messing around last night still).  It's
> intended as bit an integration test "suite" and a how-to to test newly
> released and snapshot jars.
>
>
> @rawkintrevo.. what is the status with the Zeppelin Mahout kernel?  I know
> it was never merged to the Zeppelin master, but I thought I remembered some
> integration.. is it just a question of adding Mahout jars to the spark
> kernel?  Id like to use it for ggplot2, and some separate demos.
>
> Any info is appreciated.
>
> Thanks,
>
> Andy
>

Re: 14.1 release

[Andrew Palumbo]

>> I'll OHSU it to a mahout-14.1


Should read I'll *push* it to a mahout-14.1 branch.

I'm laid up today.  Hopefully tomorrow.




On Oct 9, 2019 3:44 PM, Andrew Palumbo  wrote:

Anybody have a status on the 14.1 release?  I noticed that it was pushed to a 
mahout-14.2 branch. Bit sure if the snapshots are set at 14.2 but this release 
should be 14.1 correct?

Assuming 72 hr lazy consensus I'll OHSU it to a mahout-14.1 branch and delete 
mahout-14.2

Andy

Re: Jenkins

[Andrew Palumbo]

Yea I remember that now.  It was pushing a bunch .

Will check it out when I can.

On Oct 11, 2019 4:33 AM, Trevor Grant  wrote:

I also vaguely remember nightlies breaking and we just shut a bunch of it
off to make it shut up.

It was also publishing nightlies of weird stuff that has been out of the
trunk for 3+ years, iirc.





On Wed, Oct 9, 2019 at 5:51 PM Andrew Palumbo  wrote:

> Has anybody checked on Jenkins lately.  It doesn't seem to be pushing
> nightlies.
>
> Not sure if this was disabled intentionally I seem to remember discussion
> about this.
>
> --andy
>

Jenkins

[Andrew Palumbo]

Has anybody checked on Jenkins lately.  It doesn't seem to be pushing nightlies.

Not sure if this was disabled intentionally I seem to remember discussion about 
this.

--andy

14.1 release

[Andrew Palumbo]

Anybody have a status on the 14.1 release?  I noticed that it was pushed to a 
mahout-14.2 branch. Bit sure if the snapshots are set at 14.2 but this release 
should be 14.1 correct?

Assuming 72 hr lazy consensus I'll OHSU it to a mahout-14.1 branch and delete 
mahout-14.2

Andy

Zeppelin status

[Andrew Palumbo]

Hey all.. I'm working on some simple Mahout integration programs.. they're 
publicly available on my GitHub.

https://github.com/andrewpalumbo/mahoutTestApps

(Need to push some  hanged from messing around last night still).  It's 
intended as bit an integration test "suite" and a how-to to test newly released 
and snapshot jars.


@rawkintrevo.. what is the status with the Zeppelin Mahout kernel?  I know it 
was never merged to the Zeppelin master, but I thought I remembered some 
integration.. is it just a question of adding Mahout jars to the spark kernel?  
Id like to use it for ggplot2, and some separate demos.

Any info is appreciated.

Thanks,

Andy

[jira] [Created] (MAHOUT-2069) Typo in core/pom.xml

[Andrew Palumbo (JIRA)]

Andrew Palumbo created MAHOUT-2069:
--

 Summary: Typo in core/pom.xml
 Key: MAHOUT-2069
 URL: https://issues.apache.org/jira/browse/MAHOUT-2069
 Project: Mahout
  Issue Type: Task
Reporter: Andrew Palumbo
Assignee: Andrew Palumbo


typo in \{core/pom.xml} breaks build.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: 0.14.0 RC2

[Andrew Palumbo]

+1

Re: 0.14.0 RC2

[Andrew Palumbo]

maybe we just need to add fastutil to the shell pom?

From: Andrew Musselman 
Sent: Monday, March 4, 2019 12:19 PM
To: Mahout Dev List
Subject: Re: 0.14.0 RC2

Running the example in the README gives a class not found:
"java.lang.NoClassDefFoundError:
it/unimi/dsi/fastutil/ints/Int2DoubleOpenHashMap"

If that's just us still using something that's been removed, it's not a
deal-breaker for me as long as we fix it in a quick point release.

Pending that being a simple fix my vote is +1 binding, and if Andy's not
back from vacation and his proxy works that's +2 binding from me and Andy.


bob $ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
bob $ export MAHOUT_HOME=//home/akm/a/src/test/
repository.apache.org/content/repositories/orgapachemahout-1052/org/apache/mahout/mahout/0.14.0
bob $ export SPARK_HOME=/home/akm/a/src/spark-2.1.0-bin-hadoop2.7
bob $ MASTER=local[2] mahout-0.14.0/bin/mahout spark-shell
Adding lib/ to CLASSPATH
Using Spark's default log4j profile:
org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use
setLogLevel(newLevel).
19/03/04 09:07:44 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
19/03/04 09:07:44 WARN Utils: Your hostname, Bob resolves to a loopback
address: 127.0.1.1; using 10.0.1.2 instead (on interface eno1)
19/03/04 09:07:44 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to
another address
19/03/04 09:07:53 WARN ObjectStore: Failed to get database global_temp,
returning NoSuchObjectException
Spark context Web UI available at http://10.0.1.2:4040
Spark context available as 'sc' (master = local[2], app id =
local-1551719265339).
Spark session available as 'spark'.
Loading /home/akm/a/src/test/
repository.apache.org/content/repositories/orgapachemahout-1052/org/apache/mahout/mahout/0.14.0/mahout-0.14.0/bin/load-shell.scala.
..
import org.apache.mahout.math._
import org.apache.mahout.math.scalabindings._
import org.apache.mahout.math.drm._
import org.apache.mahout.math.scalabindings.RLikeOps._
import org.apache.mahout.math.drm.RLikeDrmOps._
import org.apache.mahout.sparkbindings._
sdc: org.apache.mahout.sparkbindings.SparkDistributedContext =
org.apache.mahout.sparkbindings.SparkDistributedContext@749ffdc7

_ _
_ __ ___   __ _| |__   ___  _   _| |_
 '_ ` _ \ / _` | '_ \ / _ \| | | | __|
 | | | | (_| | | | | (_) | |_| | |_
_| |_| |_|\__,_|_| |_|\___/ \__,_|\__|  version 0.14.0



That file does not exist

Welcome to
    __
 / __/__  ___ _/ /__
_\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 2.1.0
  /_/

Using Scala version 2.11.8 (OpenJDK 64-Bit Server VM, Java 1.8.0_191)
Type in expressions to have them evaluated.
Type :help for more information.

scala> :load /home/akm/a/src/test/
repository.apache.org/content/repositories/orgapachemahout-1052/org/apache/mahout/mahout/0.14.0/mahout-0.14.0/examples/bin/SparseSparseDrmTimer.mscala
Loading /home/akm/a/src/test/
repository.apache.org/content/repositories/orgapachemahout-1052/org/apache/mahout/mahout/0.14.0/mahout-0.14.0/examples/bin/SparseSparseDrmTimer.mscala.
..
timeSparseDRMMMul: (m: Int, n: Int, s: Int, para: Int, pctDense: Double,
seed: Long)Long

scala> timeSparseDRMMMul(1000,1000,1000,1,.02,1234L)
19/03/04 09:13:13 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 1)
java.lang.NoClassDefFoundError:
it/unimi/dsi/fastutil/ints/Int2DoubleOpenHashMap
at
org.apache.mahout.math.RandomAccessSparseVector.(RandomAccessSparseVector.java:49)
at
org.apache.mahout.math.RandomAccessSparseVector.(RandomAccessSparseVector.java:44)
at
org.apache.mahout.sparkbindings.SparkEngine$$anonfun$11$$anonfun$apply$2.apply(SparkEngine.scala:200)
at
org.apache.mahout.sparkbindings.SparkEngine$$anonfun$11$$anonfun$apply$2.apply(SparkEngine.scala:200)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.immutable.Range.foreach(Range.scala:160)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at
org.apache.mahout.sparkbindings.SparkEngine$$anonfun$11.apply(SparkEngine.scala:200)
at
org.apache.mahout.sparkbindings.SparkEngine$$anonfun$11.apply(SparkEngine.scala:195)
at scala.collection.Iterator$$anon$12.nextCur(Iterator.scala:434)
at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
at scala.collection.Iterator$class.isEmpty(Iterator.scala:330)
at scala.collection.AbstractIterator.isEmpty(Iterator.scala:1336)
at
org.apache.mahout.sparkbindings.drm.package$$anonfun$blockify$1.apply(package.scala:55)
at

[0.14.0] Release vote proxy

[Andrew Palumbo]

I am (Andrew Palumbo) hereby giving my voting proxy on the [0.14.0] release of 
Mahout to Andrew Musselman.  I will be out of the country in spotty cell 
territory for vacation.

--Andy

Re: [VOTE] Retire gene...@mahout.apache.org

[Andrew Palumbo]

+1 thanks, Trevor.

From: Shannon Quinn 
Sent: Thursday, January 24, 2019 12:48 PM
To: dev@mahout.apache.org
Subject: Re: [VOTE] Retire gene...@mahout.apache.org

+1

Had no idea that list existed.

On 1/24/19, 9:54 AM, "Andrew Musselman"  wrote:

+1

On Thu, Jan 24, 2019 at 06:53 Trevor Grant  wrote:

> A recent JIRA ticket [1] asked we add moderators to
> gene...@mahout.apache.org.
>
> Upon further investigation, the list was set up in 2010, has 9 
subscribers,
> has had about 20 emails, and all of them (except an initial 'test' email)
> were general emails to all lists.
>
> I am opening a vote to retire the list gene...@mahout.apache.org (as
> opposed to adding moderators).  The list's content will be saved, future
> emails will bounce back.
>
> The vote will remain open for 72 hours.
>
> I am officially +1 for retiring.
>
> tg
>
> [1] https://issues.apache.org/jira/browse/MAHOUT-2056
>

Re: RC1 for 0.14.0

[Andrew Palumbo]

Andrew - I was looking at this yesterday, and I only see binaries at:

https://repository.apache.org/content/repositories/orgapachemahout-1048

are there gzipped sources somewhere?

--andy

From: Andrew Musselman 
Sent: Tuesday, January 15, 2019 10:51 PM
To: Mahout Dev List
Subject: Re: RC1 for 0.14.0

Thanks Andy

On Tue, Jan 15, 2019 at 7:06 PM Andrew Palumbo  wrote:

> I'll try to get to testing again tomorrow.
>

Re: RC1 for 0.14.0

[Andrew Palumbo]

I'll try to get to testing again tomorrow.

Any changes to jenkins needed after gitbox migration?

[Andrew Palumbo]

Hello,

We've (The Mahout Project) recently moved to gitbox.  Thanks for the seamless 
integration.

I'm looking through our latest jenkins build and see that its still cloning 
from https://git-wip-us.apache.org/repos/mahout.git.

Is this the correct URL?  Is there anything necessary that we must change in 
our Jenkins builds?

In the notice I see:

>git repositories must be migrated from the git-wip-us.apache.org URL to 
>gitbox.apache.org.

will git-wip-us.apache.org remain as a mirror?

Thank you,

Andy

Mahout PMC member

Fwd: Re: [jira] [Resolved] (INFRA-17590) mahout gitbox

[Andrew Palumbo]

Re: [jira] [Resolved] (INFRA-17590) mahout gitbox

[Andrew Palumbo]

Thanks very much

Re: [NOTICE] Mandatory migration of git repositories to gitbox.apache.org

[Andrew Palumbo]

I'd like to call a vote on moving to gitbox.  Here's my +1

Re: Good Day

[Andrew Palumbo]

My fault getting this through moderation.  It was caught.

Test again

[Andrew Palumbo]

Tested

Test slack

[Andrew Palumbo]

Wake up slack

0.14.0 bulid

[Andrew Palumbo]

Hey Guys,

I was just going through the build, and noticed that some of the logging has 
canged or is missin g from what is now `core`.  i noticed in the old math-scala 
tests, I think.


SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further 
details.


Just wanted to make you aware.  I wont be able to commit much if time to this 
Release, but have a little more time on my hands than before.  wanted to make 
you aware,  my guess is that this is likely just a logginf conf issue from the 
refactor to the new conf.


This may likely is ony in the tests, and i don't hink isa an issue.


Anyways:



$ mvn clean install


[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Mahout .. SUCCESS [  1.454 s]
[INFO] Mahout Core  SUCCESS [03:02 min]
[INFO] Mahout Engine .. SUCCESS [  0.178 s]
[INFO] - Mahout HDFS Support .. SUCCESS [  3.366 s]
[INFO] - Mahout Spark Engine .. SUCCESS [01:27 min]
[INFO] Mahout Community ... SUCCESS [  0.876 s]
[INFO] - Mahout Spark CLI Drivers . SUCCESS [01:26 min]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 06:02 min
[INFO] Finished at: 2018-11-12T15:09:38-08:00
[INFO] Final Memory: 129M/1526M
[INFO] 
[branch-0.14.0]

Thanks very much for all of the hard work.

--andy

Re: [jira] [Created] (MAHOUT-2055) Implementation of Baum-Welch algorithm for HMM in Mahout Samsara

[Andrew Palumbo]

Very nice.. thanks alot!

Re: Dev List - mahout

[Andrew Palumbo]

Try going to dev-fwd or Dev list type a message into one of the threads.  It 
should pop up on dev@ from mahout.share..

Re: Dev List - mahout

[Andrew Palumbo]

Trying to hook up a forwarding both from slack -> dev@.  To get some 
conversations on the record and bump the dev@ stats.  So far it seems that I 
can get some messages to go one way..

If you go to #dev-fwd or #dev-list, right now w mail Clark it seems to go one 
way
[https://media.giphy.com/media/CaiVJuZGvR8HK/source.gif]
And is kind of a pain in the ass.  But it may work to capture slack 
conversations on dev@ until we get the php webhooks in..

test slack

[Andrew Palumbo]

test test

Test

[Andrew Palumbo]

Test

Ping slack

[Andrew Palumbo]

Test slack

Re: Friday hangout

[Andrew Palumbo]

Probably my calendar messed it up.
Thx
--andy

On Sep 3, 2018 10:32 AM, Andrew Musselman  wrote:

Huh, it's supposed to be
"Friday, September 7
9:00 – 10:00am"

On Mon, Sep 3, 2018 at 10:28 AM Andrew Palumbo  wrote:

> FYI, @andrew.. the calendar invite reads 9 am Sept 3 for me (android).
>