Ugly... I wonder if it means that Hadoop development has decided to go
full-parcel (and its irks) and make Hadoop a boutique software?

At any rate, I believe this will slower the adoption of it to a point of where
it is only available through a single vendor left in the commercial market.
But that, fortunately, not my problem to resolve ;)

Cos
 
On Thu, Jul 25, 2019 at 05:27AM, Olaf Flebbe wrote:
> Hi
> 
> hadoop changed their scripts to break when not absolutely everything is in 
> place.
> 
> The start scripts for yarn for example expect to all jars for hdfs, yarn and
> mapreduce to be present. They renamed (to be precise deprecated the old
> name) of environment variables in a way it looks more convenient, but it
> will mix up hadoop subprojects. we cannot use independent /run and /var dirs
> as the debian build rules and some other unwritten rules of linux packaging
> demand in order to be installed independently.
> 
> So hadoop has tied everything together. without rewriting their build
> scripts it is not possible to have yarn without hdfs or hdfs without yarn to
> be installed independently. 
> 
> Olaf
> 
> ps: The way pid files are written confuses systemd (or only me). effectively
> systemd somehow doesnt properly detect the state of the daemons.
> 
> Von meinem iPad gesendet
> 
> > Am 25.07.2019 um 02:11 schrieb Konstantin Boudnik <[email protected]>:
> > 
> > Olaf, thanks for trying... this could be exhausting, for sure ;(
> > 
> > Just to clarify: when you say "monolithic chunk" doesn't it mean that it
> > relies heavily on relative paths or something of the sort and couldn't be
> > broken into pieces because... well, it is so broken?
> > 
> > Thanks,
> >  Cos
> > 
> >> On Mon, Jul 22, 2019 at 10:34PM, Olaf Flebbe wrote:
> >> Hi,
> >> 
> >> If I would have only known that google document before ... I am
> >> asking myself if we should link it from confluence.
> >> 
> >> + hadoop 3.
> >> As you may know, I worked on it (bigtop-alpha branch), got it
> >> compiled and packaged. However, while testing, I discovered that
> >> Hadoop project built on top of some assumptions which do not hold
> >> true for current Bigtop. One of them is that Hadoop 3 is to be
> >> installed as a monolithic chunk on a filesystem. This is not what I
> >> understand as integrated into a Linux distribution. Other point is
> >> that hadoop's installation proposal does not follow the LSB:
> >> platform specfic libs are in "share" directories. I am exhausted,
> >> this is so broken.
> >> 
> >> I will stop here ... for a reason.
> >> 
> >> Olaf
> >> 
> >> 
> >> 
> >> 
> >> 
> >>> Am 17.07.19 um 09:09 schrieb Youngwoo Kim (김영우):
> >>> Hi folks,
> >>> 
> >>> After 1.4.0 release, there is no discussion for the next release yet. so I
> >>> believe we need to share the ideas and prioritize the items for
> >>> development.
> >>> 
> >>> And also https://issues.apache.org/jira/browse/BIGTOP-3123 and
> >>> https://docs.google.com/document/d/1F2Gxu8GARQDZXgqHn12LKkQ5wCV_AF4b_tVmjYB6YfA/edit
> >>> are good starting point for this discussion.
> >>> 
> >>> My personal preferences are:
> >>> - Distribution based on Hadoop 3
> >>> - Up-to-date BigPetStore
> >>> - Software stacks and framework for streaming data & Machine Learning
> >>> - Containers and Cloud Native: What? How?
> >>> 
> >>> It would be great to hear your thoughts.
> >>> 
> >>> Thanks,
> >>> Youngwoo
> >>> 

Attachment: signature.asc
Description: Digital signature

Reply via email to