Hi, I am trying to use Bigtop to create deb file for Hadoop 3.2.1. I am using Ubuntu 18.04.
In the process of doing this, many questions came to my mind which I wanted to learn about. 1) I changed the bom file and ran gradle hadoop-deb command. Is this the correct process to create any deb or rpm? 2) In what state you will use patches present in folder common/src/hadoop/ , is it only usable when upstream system make additional changes after making a release? 3) Many times the build fails in between and I had to clean up build and output folder both and restart. What is the process to pick from where is failed to save time? 4) Follow-up for 3), Building the hadoop and building deb for hadoop are two different things, what command can be used to do one after the another manually to save time in case one fails. > Task :hadoop-deb FAILED dpkg-source: warning: extracting unsigned source package (hadoop_3.2.1-1.dsc) dpkg-source: error: unpack target exists: hadoop-3.2.1 dpkg-source: info: extracting hadoop in hadoop-3.2.1 4) How does Bigtop ensures compatability between products, example Hive is compiled with Hadoop version 3.2.1. Based on my understanding in the mvn -D commands there is an override and version it takes from the bom file the version information. Is this understanding correct? Do I still need to change any maven pom file for any software to ensure compatibility? 5) Are there any more resources to learn about Bigtop other than https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+Packaging. What should I do next to learn more about Bigtop 6) Beside installing deb on my machine and playing with it, what else tests are needed can be done to ensure it works as intended. 7) For Hadoop 3.1.2 I see there is a exit 1 in the install_hadoop.sh My build fails around that, is there anyway to debug this. Why is this exit 1 ? https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/install_hadoop.sh#L438 + '[' -e debian/tmp//usr/lib/hadoop-mapreduce//hadoop-annotations-3.2.1.jar ']' + exit 1 debian/rules:55: recipe for target 'override_dh_auto_install' failed make[1]: *** [override_dh_auto_install] Error 1 make[1]: Leaving directory '/home/jj/dev/code/open/bigtop/output/hadoop/hadoop-3.2.1' debian/rules:27: recipe for target 'binary' failed make: *** [binary] Error 2 dpkg-buildpackage: error: fakeroot debian/rules binary subprocess returned exit status 2 debuild: fatal error at line 1152: dpkg-buildpackage -rfakeroot -us -uc -ui -b failed Thanks in advance
