I think the issue is coming from TestDbTxnManager test. This test generates a huge hive.log file (of the order of 25-30 GB). Created HIVE-16109 to fix this. We should cleanup older logs manually to free the disk space meanwhile so that pre-commit job is unblocked again.
On Fri, Mar 3, 2017 at 12:29 PM, Sergio Pena <sergio.p...@cloudera.com> wrote: > Hey Vihang, > > Seems there is a UT test causing all this noise. See this: > > 30G ./PreCommit-HIVE-Build-3902/failed/272_UTBatch_ql_10_ > tests/logs/hive.log > 30G ./PreCommit-HIVE-Build-3900/failed/272_UTBatch_ql_10_ > tests/logs/hive.log > 30G ./PreCommit-HIVE-Build-3899/failed/272_UTBatch_ql_10_ > tests/logs/hive.log > 30G ./PreCommit-HIVE-Build-3898/failed/272_UTBatch_ql_10_ > tests/logs/hive.log > 30G ./PreCommit-HIVE-Build-3897/failed/272_UTBatch_ql_10_ > tests/logs/hive.log > 30G ./PreCommit-HIVE-Build-3896/failed/272_UTBatch_ql_10_ > tests/logs/hive.log > 30G ./PreCommit-HIVE-Build-3893/failed/272_UTBatch_ql_10_ > tests/logs/hive.log > 30G ./PreCommit-HIVE-Build-3892/failed/272_UTBatch_ql_10_ > tests/logs/hive.log > 30G ./PreCommit-HIVE-Build-3891/failed/272_UTBatch_ql_10_ > tests/logs/hive.log > 30G ./PreCommit-HIVE-Build-3890/failed/272_UTBatch_ql_10_ > tests/logs/hive.log > 30G ./PreCommit-HIVE-Build-3888/failed/272_UTBatch_ql_10_ > tests/logs/hive.log > 30G ./PreCommit-HIVE-Build-3887/failed/272_UTBatch_ql_10_ > tests/logs/hive.log > 30G ./PreCommit-HIVE-Build-3877/failed/272_UTBatch_ql_10_ > tests/logs/hive.log > 29G ./PreCommit-HIVE-Build-3903/failed/272_UTBatch_ql_10_ > tests/logs/hive.log > 29G ./PreCommit-HIVE-Build-3894/failed/272_UTBatch_ql_10_ > tests/logs/hive.log > 26G ./PreCommit-HIVE-Build-3904/failed/272_UTBatch_ql_10_ > tests/logs/hive.log > 26G ./PreCommit-HIVE-Build-3876/failed/272_UTBatch_ql_10_ > tests/logs/hive.log > 24G ./PreCommit-HIVE-Build-3895/failed/272_UTBatch_ql_10_ > tests/logs/hive.log > 19G ./PreCommit-HIVE-Build-3918/failed/272_UTBatch_ql_10_ > tests/logs/.hive.log.9uk9Av > 5.1G ./PreCommit-HIVE-Build-3913/failed/14-TestCliDriver- > authorization_create_temp_table.q-skewjoinopt16.q-drop_ > partitions_filter3.q-and-27-more/logs/hive.log > 4.8G ./PreCommit-HIVE-Build-3913/failed/20-TestCliDriver- > rename_column.q-union5.q-insert_into1.q-and-27-more/logs/hive.log > 4.7G ./PreCommit-HIVE-Build-3913/failed/19-TestCliDriver-cp_mj_ > rc.q-order.q-udf_bitwise_shiftleft.q-and-27-more/logs/hive.log > 4.0G ./PreCommit-HIVE-Build-3918/succeeded/185_UTBatch_service_ > 8_tests/logs/hive.log > 4.0G ./PreCommit-HIVE-Build-3917/succeeded/185_UTBatch_service_ > 8_tests/logs/hive.log > > You can see all the logs from this link: > http://104.198.109.242/logs/ > > > > On Fri, Mar 3, 2017 at 2:21 PM, Sergio Pena <sergio.p...@cloudera.com> > wrote: > >> Hey Vihang, >> >> The ptest master shows how the jobs has increased the log size, could you >> investigate what causes this increase? >> >> 44G PreCommit-HIVE-Build-3902 >> 44G PreCommit-HIVE-Build-3900 >> 44G PreCommit-HIVE-Build-3899 >> 44G PreCommit-HIVE-Build-3898 >> 44G PreCommit-HIVE-Build-3897 >> 44G PreCommit-HIVE-Build-3896 >> 44G PreCommit-HIVE-Build-3893 >> 44G PreCommit-HIVE-Build-3892 >> 44G PreCommit-HIVE-Build-3891 >> 44G PreCommit-HIVE-Build-3890 >> 44G PreCommit-HIVE-Build-3888 >> 44G PreCommit-HIVE-Build-3877 >> 43G PreCommit-HIVE-Build-3903 >> 43G PreCommit-HIVE-Build-3894 >> 41G PreCommit-HIVE-Build-3887 >> 40G PreCommit-HIVE-Build-3904 >> 40G PreCommit-HIVE-Build-3876 >> 38G PreCommit-HIVE-Build-3895 >> 16G PreCommit-HIVE-Build-3918 >> 15G PreCommit-HIVE-Build-3917 >> 15G PreCommit-HIVE-Build-3916 >> 15G PreCommit-HIVE-Build-3915 >> 15G PreCommit-HIVE-Build-3913 >> 15G PreCommit-HIVE-Build-3912 >> 15G PreCommit-HIVE-Build-3911 >> 15G PreCommit-HIVE-Build-3910 >> 15G PreCommit-HIVE-Build-3909 >> 15G PreCommit-HIVE-Build-3905 >> 14G PreCommit-HIVE-Build-3914 >> 14G PreCommit-HIVE-Build-3878 >> 212K PreCommit-HIVE-Build-3889 >> 212K PreCommit-HIVE-Build-3880 >> 188K PreCommit-HIVE-Build-3901 >> 156K PreCommit-HIVE-Build-3885 >> 148K PreCommit-HIVE-Build-3884 >> 96K PreCommit-HIVE-Build-3882 >> 88K PreCommit-HIVE-Build-3886 >> 84K PreCommit-HIVE-Build-3881 >> 80K PreCommit-HIVE-Build-3883 >> 60K PreCommit-HIVE-Build-3879 >> 56K PreCommit-HIVE-Build-3908 >> >> - Sergio >> >> On Fri, Mar 3, 2017 at 12:03 PM, Vihang Karajgaonkar <vih...@cloudera.com >> > wrote: >> >>> Hi Sergio, >>> >>> Looks like the pre-commit jobs are failing due to disk space issues. Can >>> you please help or let me know how can I fix it? >>> >>> Thanks, >>> Vihang >>> >> >> >
