Hi hadoop changed their scripts to break when not absolutely everything is in place.
The start scripts for yarn for example expect to all jars for hdfs, yarn and mapreduce to be present. They renamed (to be precise deprecated the old name) of environment variables in a way it looks more convenient, but it will mix up hadoop subprojects. we cannot use independent /run and /var dirs as the debian build rules and some other unwritten rules of linux packaging demand in order to be installed independently. So hadoop has tied everything together. without rewriting their build scripts it is not possible to have yarn without hdfs or hdfs without yarn to be installed independently. Olaf ps: The way pid files are written confuses systemd (or only me). effectively systemd somehow doesnt properly detect the state of the daemons. Von meinem iPad gesendet > Am 25.07.2019 um 02:11 schrieb Konstantin Boudnik <[email protected]>: > > Olaf, thanks for trying... this could be exhausting, for sure ;( > > Just to clarify: when you say "monolithic chunk" doesn't it mean that it > relies heavily on relative paths or something of the sort and couldn't be > broken into pieces because... well, it is so broken? > > Thanks, > Cos > >> On Mon, Jul 22, 2019 at 10:34PM, Olaf Flebbe wrote: >> Hi, >> >> If I would have only known that google document before ... I am >> asking myself if we should link it from confluence. >> >> + hadoop 3. >> As you may know, I worked on it (bigtop-alpha branch), got it >> compiled and packaged. However, while testing, I discovered that >> Hadoop project built on top of some assumptions which do not hold >> true for current Bigtop. One of them is that Hadoop 3 is to be >> installed as a monolithic chunk on a filesystem. This is not what I >> understand as integrated into a Linux distribution. Other point is >> that hadoop's installation proposal does not follow the LSB: >> platform specfic libs are in "share" directories. I am exhausted, >> this is so broken. >> >> I will stop here ... for a reason. >> >> Olaf >> >> >> >> >> >>> Am 17.07.19 um 09:09 schrieb Youngwoo Kim (김영우): >>> Hi folks, >>> >>> After 1.4.0 release, there is no discussion for the next release yet. so I >>> believe we need to share the ideas and prioritize the items for >>> development. >>> >>> And also https://issues.apache.org/jira/browse/BIGTOP-3123 and >>> https://docs.google.com/document/d/1F2Gxu8GARQDZXgqHn12LKkQ5wCV_AF4b_tVmjYB6YfA/edit >>> are good starting point for this discussion. >>> >>> My personal preferences are: >>> - Distribution based on Hadoop 3 >>> - Up-to-date BigPetStore >>> - Software stacks and framework for streaming data & Machine Learning >>> - Containers and Cloud Native: What? How? >>> >>> It would be great to hear your thoughts. >>> >>> Thanks, >>> Youngwoo >>>
