I suspect the bigger issue is that the hbase tests are running on the
‘ubuntu’ machines. Since they only have ~300GB for workspaces, the hbase tests
are eating a significant majority of it and likely could be dying randomly due
to space issues. [All the hbase workspace directories + the yetus-m2 shared
mvn cache dirs easily consume 20%+ of the space. Significantly more than the 50
or so other jobs that run on those machines.]
By comparison, most of the ‘Hadoop’ nodes have 2-3TB for the big jobs
to consume….
> On Jul 24, 2018, at 8:58 AM, Josh Elser <[email protected]> wrote:
>
> Yep, sadly this is a very long tent-pole for us. There are many involved who
> have invested countless hours in making this better.
>
> Specific to that job you linked earlier, 3 test failures out of our total
> 4958 tests (0.06% failure rate) is all but "green" in my mind. I would ask
> that you keep that in mind, too.
>
> To that extent, others have also built another job specifically to find tests
> which are failing intermittently:
> https://builds.apache.org/job/HBase-Find-Flaky-Tests/25513/artifact/dashboard.html.
> I mention this as evidence to prove to you that this is not a baseless
> request from the HBase PMC ;)
>
> On 7/24/18 3:14 AM, Gav wrote:
>> Ok, good enough, will wait, please also note 'master' branch and a few
>> others have been failing for over a month also.
>> I will check in again next month to see how things are progressing
>> Thanks
>> Gav...
>> On Tue, Jul 24, 2018 at 1:19 AM Josh Elser <[email protected]> wrote:
>>> Hi Gav,
>>>
>>> Looking at the most recent results, I see that the job failed because of
>>> two unit test failures. These are something that will be looked at prior
>>> to the next 1.4.x release which is about to get off the ground.
>>>
>>> I'd kindly request that you not disable the job. Thanks for trying to
>>> find extra resources on these nodes.
>>>
>>> On 7/23/18 12:22 AM, Gavin McDonald wrote:
>>>> https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/
>>>>
>>>> can someone take a look into this, the job isnt much good if it is
>>> failing
>>>> all the time and even worse if it is being ignored.
>>>>
>>>> Otherwise I'll disable the job in a dew days to release these wasted
>>>> resources
>>>> to builds that matter.
>>>>
>>>>
>>>
