Sorry ...
didn't see your response ... into Mahout too deep ;-)
Guess we have to postpone this to sometime over the week.
Chris
Am 19.04.20, 19:56 schrieb "Andrew Musselman" <andrew.mussel...@gmail.com>:
*what time
On Sun, Apr 19, 2020 at 10:51 Andrew Musselman <andrew.mussel...@gmail.com>
wrote:
> I think it's safe to move to 1.8; yeah what tune is good for you? I'm in
> Pacific time zone and am flexible this afternoon.
>
> Trevor you free?
>
> On Sun, Apr 19, 2020 at 09:37 Christofer Dutz <christofer.d...@c-ware.de>
> wrote:
>
>> Yikes! ... well guess then I can't help you folks as it's almost
>> impossible to get my hands on a 1.7 version.
>>
>> What's preventing you from going to 1.8+?
>>
>> Any your build says 1.8 and above:
>> <requireJavaVersion>
>> <version>[1.8,)</version>
>> </requireJavaVersion>
>>
>> Regarding the artifacts ... would it be ok to have the maven artifacts
>> using classifiers and perhaps the files in the libs and distribution to
>> follow the typical scala sheme?
>>
>> Chris
>>
>> Am 19.04.20, 17:56 schrieb "Trevor Grant" <trevor.d.gr...@gmail.com>:
>>
>> Yea, we have a requirement on 1.7. we need to get it up to 8, but
>> considered that a different issue.
>>
>> Maven throws warning, sbt breaks down entirely (when importing)-
>> hence why
>> we were using a script to replace 2.11 w 2.12
>>
>> On Sun, Apr 19, 2020, 10:33 AM Christofer Dutz <
>> christofer.d...@c-ware.de>
>> wrote:
>>
>> > Well I have been compiling with various jdks from 8 to 14. However
I
>> > noticed that if I select a jdk and start the build that the Scala
>> parts
>> > Seefahrt to the version in my systems path. Which then results in
>> all sorts
>> > of major version problems.
>> >
>> > For the Scalatest plugin I managed to have it use the maven
>> java.home...
>> > The Scala compiler is being a little trickyer.
>> >
>> > Will continue this evening. Keep your fingers crossed.
>> >
>> > Also does maven totally not like variables in artifactids.
>> >
>> > Chris
>> > ________________________________
>> > Von: Trevor Grant <trevor.d.gr...@gmail.com>
>> > Gesendet: Sonntag, 19. April 2020 15:41
>> > An: Mahout Dev List <dev@mahout.apache.org>
>> > Betreff: Re: Hi ... need some help?
>> >
>> > Hey Chris,
>> >
>> > First of all- thank you so much for your work so far.
>> >
>> > The build 100% currently works, the only issue is with releasing
>> binaries.
>> > Are you compiling w 1.7? Not really sure why artifact IDs are
>> changing?
>> > That's very strange.
>> >
>> > To answer your question regarding the community block- no, it is
>> not as if
>> > there is a company that pays us to work on Mahout and the rest is
>> > community. What it means is that while we appreciate those code
>> > contributions, no one still actively involved is maintaining them
>> and we
>> > advise people to use at their own risk. The refactor to a
>> `community`
>> > folder was meant to reflect the reality of the situation.
>> >
>> > The only test issues I was aware of were with Javadocs when trying
>> to do a
>> > release with Java 1.8+ those would fail due to formatting.
>> >
>> > I feel like you're hitting some unique issues, and maybe we could
>> help you
>> > out / there may exist simple solutions.
>> >
>> > Thank you again for your help!
>> >
>> > tg
>> >
>> >
>> >
>> >
>> > On Sun, Apr 19, 2020 at 2:17 AM Christofer Dutz <
>> christofer.d...@c-ware.de
>> > >
>> > wrote:
>> >
>> > > Hi folks,
>> > >
>> > > So yesterday I invested several hours in cleaning up your build.
>> I got
>> > > quite far, currently trying to get the tests to pass. For some
>> > > Scala-Java-major-version problems, but working on fixing them.
>> > >
>> > > However sind things will be different. I hope that's OK.
>> Especially with
>> > > the artifact ids.
>> > >
>> > > Just as a question: does the build currently work at all?
>> Especially in
>> > > the community block I had to fix quite a few API changes where
>> the code
>> > in
>> > > the blocks were for older versions of libraries.
>> > >
>> > > I hope I get some more tests to pass today.
>> > >
>> > > Chris
>> > > ________________________________
>> > > Von: Andrew Musselman <a...@apache.org>
>> > > Gesendet: Donnerstag, 16. April 2020 20:35
>> > > An: Mahout Dev List <dev@mahout.apache.org>
>> > > Betreff: Re: Hi ... need some help?
>> > >
>> > > Looking forward to working on this with you; thanks again!
>> > >
>> > > On Thu, Apr 16, 2020 at 11:09 AM Christofer Dutz <
>> > > christofer.d...@c-ware.de>
>> > > wrote:
>> > >
>> > > > Hi Andrew,
>> > > >
>> > > > guess I'll start with the fork and contact you folks on slack.
>> > > >
>> > > > Chris
>> > > >
>> > > >
>> > > >
>> > > > Am 16.04.20, 19:43 schrieb "Andrew Musselman"
<a...@apache.org>:
>> > > >
>> > > > Chris, thank you for your help..
>> > > >
>> > > > Yeah if you fork what's in master you can see what state
>> it's in;
>> > we
>> > > > are in
>> > > > the #mahout channel in tge-asf slack and this is also a
>> fine way to
>> > > > keep
>> > > > track of discussion.
>> > > >
>> > > > We could file a JIRA ticket as well, however you prefer to
>> work.
>> > > >
>> > > > Best
>> > > > Andrew
>> > > >
>> > > > On Thu, Apr 16, 2020 at 06:59 Christofer Dutz <
>> > > > christofer.d...@c-ware.de>
>> > > > wrote:
>> > > >
>> > > > > Hi Trevor,
>> > > > >
>> > > > > ok ... first of all ... the Mahout PMC is defining a
>> "community
>> > > > > maintained" library which is not maintained by the mahout
>> PMC?!?!
>> > > > > I thought at Apache everything is about Community over
>> code. So
>> > is
>> > > a
>> > > > > company driving the non-community stuff?
>> > > > >
>> > > > > But back to your build issues:
>> > > > > I had a look and I too encountered these comments and
>> remarks and
>> > > > > sometimes patterns I recognized and could imagine why
>> they were
>> > > > created.
>> > > > > Yes quite a bit of the build could be cleaned up and
>> simplified a
>> > > > lot.
>> > > > >
>> > > > > So how about I create a fork and try to do a cleanup of
>> the
>> > build.
>> > > > > Usually I also leave comments about what I do as I hope
>> I'll not
>> > be
>> > > > the
>> > > > > only one maintaining a build and documenting things helps
>> people
>> > > > feel more
>> > > > > confident.
>> > > > >
>> > > > > However in some cases I will have questions ... so would
>> someone
>> > be
>> > > > > available on Slack for quick questions?
>> > > > >
>> > > > > Usually switching to another build system does solve some
>> > problems
>> > > > ...
>> > > > > mostly the reason to switch is that it solved the main
>> problem
>> > that
>> > > > you are
>> > > > > having with the old.
>> > > > > However you usually notice too late that you get yourself
>> a lot
>> > of
>> > > > new
>> > > > > problems. I remember doing some contract work for an
>> insurance
>> > > > company and
>> > > > > they were totally down Maven-road but then had to build
>> something
>> > > > with SBT
>> > > > > ... in the end I compiled the thing on my laptop, copied
>> it to a
>> > > USB
>> > > > stick
>> > > > > and told the people what was on the stick and that I'll
be
>> > having a
>> > > > coffee
>> > > > > and will be back in 30 minutes. When I came back the sick
>> wasn't
>> > at
>> > > > the
>> > > > > same place and the build problem was "solved" ;-)
>> > > > >
>> > > > > So I think it's quite good to stick to maven ... that is
>> very
>> > > > mature, you
>> > > > > can do almost everything you want with it and it
>> integrates
>> > > > perfectly into
>> > > > > the Apache infrastructure.
>> > > > >
>> > > > > But that's just my opinion.
>> > > > >
>> > > > > So if you want me to help, I'll be happy to be of
>> assistance.
>> > > > >
>> > > > >
>> > > > > Chris
>> > > > >
>> > > > >
>> > > > >
>> > > > > Am 16.04.20, 15:28 schrieb "Trevor Grant" <
>> > > trevor.d.gr...@gmail.com
>> > > > >:
>> > > > >
>> > > > > Hey Christopher,
>> > > > >
>> > > > > I would agree with what Stevo outlined but add some
>> more
>> > > context
>> > > > and a
>> > > > > couple related JIRA issues.
>> > > > >
>> > > > > For 0.14.0 We did a big refactor and finally moved
the
>> > > MapReduce
>> > > > based
>> > > > > Mahout all into what we called "community/" that is
>> community
>> > > > > maintained,
>> > > > > which is to say, we're not maintaining it anymore
>> (sunset
>> > > began I
>> > > > > think in
>> > > > > 2015).
>> > > > >
>> > > > > But all of our POMs were so huge and fat because
>> they'd been
>> > > > layered up
>> > > > > over the years by people coming and going and
>> dropping in
>> > > code. I
>> > > > > wouldn't
>> > > > > call these drive- bys, its just been over 10 years
>> and people
>> > > > come and
>> > > > > go.
>> > > > > Such is the life of Apache Projects. So we had a
>> situation
>> > > where
>> > > > a lot
>> > > > > of
>> > > > > the old Map Reduce stuff and the POMs were considered
>> > > > "old-magic" no
>> > > > > one
>> > > > > really knew how it was all tied together, but we
>> didn't want
>> > to
>> > > > mess
>> > > > > with
>> > > > > it for fear of breaking something in the "new" Mahout
>> (aka
>> > > > Samsar)
>> > > > > which is
>> > > > > the Scala/Spark based library that it is now* (to
>> others in
>> > the
>> > > > > community:
>> > > > > I know it runs on other engines, but for simplicity,
>> I'm just
>> > > > calling
>> > > > > it
>> > > > > "runs-on-spark").
>> > > > >
>> > > > > For 0.14.0 We decided to trim out as much of that
>> which was
>> > > > possible.
>> > > > > We
>> > > > > did some major liposuction on POMs, re organized
>> things, etc.
>> > > > This was
>> > > > > done
>> > > > > by commenting out a section, then seeing if it would
>> still
>> > > > build. So
>> > > > > the
>> > > > > current release
>> > > > > _does_ build. And aside for some CLI driver issues
>> which are
>> > > > outlined
>> > > > > in
>> > > > > [1], the project runs fairly smooth. (An SBT would
>> probably
>> > > > solve [1],
>> > > > > I
>> > > > > believe Pat Ferrel has made his own SBT script to
>> compile
>> > > > Mahout, which
>> > > > > solved that problem for them).
>> > > > >
>> > > > > The issue we ran into with the releases (and the
>> reason I
>> > think
>> > > > you're
>> > > > > here), is that we also somewhere along the line
>> commented out
>> > > > something
>> > > > > that was important to the release process. Hence why
>> 0.14.0
>> > > > released
>> > > > > source
>> > > > > only.
>> > > > >
>> > > > > Since 2008, there has been a lot of great work on
>> generating
>> > > > plugins
>> > > > > for
>> > > > > doing Apache releases. Instead of the awkward hacks
>> that made
>> > > up
>> > > > the
>> > > > > old
>> > > > > poms (literally comments that said, "this is a hack,
>> there's
>> > > > supposedly
>> > > > > something better coming from ..." dated like 2012),
>> we would
>> > > > like to
>> > > > > do it
>> > > > > the "right way" and incorporate the appropriate
>> plugins.
>> > > > >
>> > > > > Refactoring to SBT was _one_ proposed solution. We're
>> also OK
>> > > > > continuing to
>> > > > > use Maven, and I agree with what you said about the
>> cross
>> > > > compiling. We
>> > > > > actually have a script that just changes the scala
>> version.
>> > We
>> > > > tried
>> > > > > using
>> > > > > the classifiers but there were issues in SBT, but the
>> way
>> > > you're
>> > > > > proposing
>> > > > > sounds a lot more pro than the route we were trying
>> for.
>> > > > >
>> > > > > That said- we'd be OK just releasing one scala/spark
>> version
>> > at
>> > > > a time.
>> > > > > But getting the convenience binaries to
>> release/publish would
>> > > be
>> > > > a
>> > > > > major
>> > > > > first step.
>> > > > >
>> > > > > Also, we really appreciate the help,
>> > > > >
>> > > > > tg
>> > > > >
>> > > > >
>> > > > > [1]
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
https://issues.apache.org/jira/projects/MAHOUT/issues/MAHOUT-2093?filter=allopenissues
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Thu, Apr 16, 2020 at 4:50 AM Christofer Dutz <
>> > > > > christofer.d...@c-ware.de>
>> > > > > wrote:
>> > > > >
>> > > > > > Hi Stevo,
>> > > > > >
>> > > > > > so let me summarize what I understood:
>> > > > > >
>> > > > > > - There are some modules in mahout that are built
>> with
>> > Scala,
>> > > > some
>> > > > > with
>> > > > > > java and some with both (At least that's what I see
>> when
>> > > > checking
>> > > > > out the
>> > > > > > project)
>> > > > > > - The current build uses Scala 2.11 to build the
>> Scala
>> > code.
>> > > > > > - The resulting libraries are only compatible with
>> Scala
>> > 2.11
>> > > > > >
>> > > > > > Now you want to also publish versions compatible
>> with Scala
>> > > > 2.12?
>> > > > > >
>> > > > > > If that's the case I think Maven could easily add
>> multiple
>> > > > executions
>> > > > > > where each compile compiles to different output
>> > directories:
>> > > > > > - Java --> target/classes
>> > > > > > - Scala 2.11 --> target/classes-2.11
>> > > > > > - Scala 2.12 --> target/classes-2.12
>> > > > > >
>> > > > > > Then the packaging would also need a second
>> execution ...
>> > > each
>> > > > of the
>> > > > > > executions bundling the classes and the
>> corresponding scala
>> > > > output.
>> > > > > > Ideally I would probably use maven classifiers to
>> > distinguish
>> > > > the
>> > > > > > artifacts.
>> > > > > >
>> > > > > > <dependency>
>> > > > > > <groupId>org.apache.mahout</groupId>
>> > > > > > <artifactId>mahout-spark</artifactId>
>> > > > > > <version>14.1-SNAPSHOT</version>
>> > > > > > <classifier>2.11</classifier>
>> > > > > > </dependency>
>> > > > > >
>> > > > > > Then it should all work in a normal maven build. In
>> the
>> > > > > distributions you
>> > > > > > could also filter the versions according to their
>> > > classifiers.
>> > > > > >
>> > > > > > So if this is the case, I could help you with this.
>> > > > > >
>> > > > > > Chris
>> > > > > >
>> > > > > >
>> > > > > > Am 16.04.20, 09:39 schrieb "Stevo Slavić" <
>> > ssla...@gmail.com
>> > > >:
>> > > > > >
>> > > > > > Disclaimer: I'm not active Mahout maintainer
>> for quite
>> > a
>> > > > while,
>> > > > > have
>> > > > > > some
>> > > > > > historical perspective, take it with a grain of
>> salt,
>> > > > could be
>> > > > > I'm
>> > > > > > missing
>> > > > > > the whole point you were approached for by a
>> wide
>> > margin
>> > > of
>> > > > > error.
>> > > > > >
>> > > > > > At a point Mahout, some of its modules, have
>> turned
>> > into
>> > > a
>> > > > scala
>> > > > > > library, and there was need to cross publish
>> those
>> > > modules,
>> > > > > across
>> > > > > > different scala versions. Back than Maven scala
>> plugin
>> > > > didn't
>> > > > > support
>> > > > > > cross
>> > > > > > publishing, it doesn't fit well with Maven's
>> build
>> > > > lifecycle
>> > > > > concept
>> > > > > > (multiple compile phases - one for each scala
>> version,
>> > > and
>> > > > what
>> > > > > not
>> > > > > > would
>> > > > > > be needed). Switching to sbt could have solved
>> the
>> > > problem.
>> > > > > Switch was
>> > > > > > deemed to be too big task, even though ages
>> have been
>> > > > spent on
>> > > > > trying
>> > > > > > to
>> > > > > > apply Maven (profiles) + bash scripts and what
>> not to
>> > > > solve the
>> > > > > > problem.
>> > > > > > Trying to apply same approach over and over
>> again and
>> > > > expecting
>> > > > > > different
>> > > > > > results is not smart, no expert can help there.
>> Mahout
>> > > > > maintainers and
>> > > > > > contributors, should consider alternative
>> approach, one
>> > > of
>> > > > them
>> > > > > being
>> > > > > > switching to sbt - it's scala native, supports
>> scala
>> > > cross
>> > > > > publishing,
>> > > > > > supports publishing Maven compatible release
>> metadata
>> > and
>> > > > > binaries.
>> > > > > >
>> > > > > > Kind regards,
>> > > > > > Stevo Slavic.
>> > > > > >
>> > > > > > On Thu, Apr 16, 2020 at 9:15 AM Christofer Dutz
>> <
>> > > > > > christofer.d...@c-ware.de>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hi folks,
>> > > > > > >
>> > > > > > > my name is Chris and I’m involved in quite a
>> lot of
>> > > > Apache
>> > > > > projects.
>> > > > > > > Justin approached me this morning, asking me
>> if I
>> > could
>> > > > > perhaps help
>> > > > > > you.
>> > > > > > > He told me you were having trouble with doing
>> Maven
>> > > > releases.
>> > > > > > >
>> > > > > > > As Maven releases are my specialty, could you
>> please
>> > > > summarize
>> > > > > the
>> > > > > > issues
>> > > > > > > you are having?
>> > > > > > >
>> > > > > > > Chris
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > >
>> > > >
>> > >
>> >
>>
>>
