----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64348/#review192898 -----------------------------------------------------------
Ship it! Ship It! - Dmytro Grinenko On Dec. 5, 2017, 6:33 p.m., Jonathan Hurley wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64348/ > ----------------------------------------------------------- > > (Updated Dec. 5, 2017, 6:33 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole. > > > Bugs: AMBARI-22598 > https://issues.apache.org/jira/browse/AMBARI-22598 > > > Repository: ambari > > > Description > ------- > > When Pig (and other hadoop clients) are upgraded by themselves, they need to > create jobs with versions of the LZO libraries which match _their_ versions. > > *STR* > # Deployed cluster with Ambari version: 2.5.1.0-159 and HDP version: > 2.6.1.0-129 (HDFS has lzo enabled) > # Upgrade Ambari to Target Version: 2.6.1.0-84 > # Perform full stack upgrade to 2.6.4.0-55 > # Perform 4th digit PU of some services in batches (see attached for list) > # Run Pig service check > > *Result* > ``` > Caused by: java.lang.IllegalArgumentException: Compression codec > com.hadoop.compression.lzo.LzoCodec not found. > at > org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:139) > at > org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:180) > at > org.apache.hadoop.mapreduce.lib.input.TextInputFormat.isSplitable(TextInputFormat.java:59) > at > org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:399) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:265) > ... 28 more > Caused by: java.lang.ClassNotFoundException: Class > com.hadoop.compression.lzo.LzoCodec not found > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2147) > at > org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:132) > ... 32 more > 2017-12-04 12:07:30,322 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR > 2118: Compression codec com.hadoop.compression.lzo.LzoCodec not found. > Details at logfile: /home/ambari-qa/pig_1512389247042.log > 2017-12-04 12:07:30,338 [main] INFO org.apache.pig.Main - Pig script > completed in 3 seconds and 457 milliseconds (3457 ms) > 2017-12-04 12:07:30,338 [main] INFO > org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down > thread pool > ``` > > > Diffs > ----- > > > ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/mahout.py > f2c3c18077 > > ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/pig.py > 83f70488a3 > > ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_spark.py > 426509cb0e > > ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/setup_spark.py > 82b6c63d6d > > > Diff: https://reviews.apache.org/r/64348/diff/1/ > > > Testing > ------- > > OK > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 01:29 min > [INFO] Finished at: 2017-12-05T13:27:57-05:00 > [INFO] Final Memory: 21M/619M > [INFO] > ------------------------------------------------------------------------ > > > Thanks, > > Jonathan Hurley > >
