Hey Christopher, I would agree with what Stevo outlined but add some more context and a couple related JIRA issues.
For 0.14.0 We did a big refactor and finally moved the MapReduce based Mahout all into what we called "community/" that is community maintained, which is to say, we're not maintaining it anymore (sunset began I think in 2015). But all of our POMs were so huge and fat because they'd been layered up over the years by people coming and going and dropping in code. I wouldn't call these drive- bys, its just been over 10 years and people come and go. Such is the life of Apache Projects. So we had a situation where a lot of the old Map Reduce stuff and the POMs were considered "old-magic" no one really knew how it was all tied together, but we didn't want to mess with it for fear of breaking something in the "new" Mahout (aka Samsar) which is the Scala/Spark based library that it is now* (to others in the community: I know it runs on other engines, but for simplicity, I'm just calling it "runs-on-spark"). For 0.14.0 We decided to trim out as much of that which was possible. We did some major liposuction on POMs, re organized things, etc. This was done by commenting out a section, then seeing if it would still build. So the current release _does_ build. And aside for some CLI driver issues which are outlined in [1], the project runs fairly smooth. (An SBT would probably solve [1], I believe Pat Ferrel has made his own SBT script to compile Mahout, which solved that problem for them). The issue we ran into with the releases (and the reason I think you're here), is that we also somewhere along the line commented out something that was important to the release process. Hence why 0.14.0 released source only. Since 2008, there has been a lot of great work on generating plugins for doing Apache releases. Instead of the awkward hacks that made up the old poms (literally comments that said, "this is a hack, there's supposedly something better coming from ..." dated like 2012), we would like to do it the "right way" and incorporate the appropriate plugins. Refactoring to SBT was _one_ proposed solution. We're also OK continuing to use Maven, and I agree with what you said about the cross compiling. We actually have a script that just changes the scala version. We tried using the classifiers but there were issues in SBT, but the way you're proposing sounds a lot more pro than the route we were trying for. That said- we'd be OK just releasing one scala/spark version at a time. But getting the convenience binaries to release/publish would be a major first step. Also, we really appreciate the help, tg [1] https://issues.apache.org/jira/projects/MAHOUT/issues/MAHOUT-2093?filter=allopenissues On Thu, Apr 16, 2020 at 4:50 AM Christofer Dutz <christofer.d...@c-ware.de> wrote: > Hi Stevo, > > so let me summarize what I understood: > > - There are some modules in mahout that are built with Scala, some with > java and some with both (At least that's what I see when checking out the > project) > - The current build uses Scala 2.11 to build the Scala code. > - The resulting libraries are only compatible with Scala 2.11 > > Now you want to also publish versions compatible with Scala 2.12? > > If that's the case I think Maven could easily add multiple executions > where each compile compiles to different output directories: > - Java --> target/classes > - Scala 2.11 --> target/classes-2.11 > - Scala 2.12 --> target/classes-2.12 > > Then the packaging would also need a second execution ... each of the > executions bundling the classes and the corresponding scala output. > Ideally I would probably use maven classifiers to distinguish the > artifacts. > > <dependency> > <groupId>org.apache.mahout</groupId> > <artifactId>mahout-spark</artifactId> > <version>14.1-SNAPSHOT</version> > <classifier>2.11</classifier> > </dependency> > > Then it should all work in a normal maven build. In the distributions you > could also filter the versions according to their classifiers. > > So if this is the case, I could help you with this. > > Chris > > > Am 16.04.20, 09:39 schrieb "Stevo Slavić" <ssla...@gmail.com>: > > Disclaimer: I'm not active Mahout maintainer for quite a while, have > some > historical perspective, take it with a grain of salt, could be I'm > missing > the whole point you were approached for by a wide margin of error. > > At a point Mahout, some of its modules, have turned into a scala > library, and there was need to cross publish those modules, across > different scala versions. Back than Maven scala plugin didn't support > cross > publishing, it doesn't fit well with Maven's build lifecycle concept > (multiple compile phases - one for each scala version, and what not > would > be needed). Switching to sbt could have solved the problem. Switch was > deemed to be too big task, even though ages have been spent on trying > to > apply Maven (profiles) + bash scripts and what not to solve the > problem. > Trying to apply same approach over and over again and expecting > different > results is not smart, no expert can help there. Mahout maintainers and > contributors, should consider alternative approach, one of them being > switching to sbt - it's scala native, supports scala cross publishing, > supports publishing Maven compatible release metadata and binaries. > > Kind regards, > Stevo Slavic. > > On Thu, Apr 16, 2020 at 9:15 AM Christofer Dutz < > christofer.d...@c-ware.de> > wrote: > > > Hi folks, > > > > my name is Chris and I’m involved in quite a lot of Apache projects. > > Justin approached me this morning, asking me if I could perhaps help > you. > > He told me you were having trouble with doing Maven releases. > > > > As Maven releases are my specialty, could you please summarize the > issues > > you are having? > > > > Chris > > > >
