[
https://issues.apache.org/jira/browse/HADOOP-17873?focusedWorklogId=644014&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-644014
]
ASF GitHub Bot logged work on HADOOP-17873:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 31/Aug/21 12:52
Start Date: 31/Aug/21 12:52
Worklog Time Spent: 10m
Work Description: steveloughran commented on pull request #3341:
URL: https://github.com/apache/hadoop/pull/3341#issuecomment-909207833
> they are the same across different tests/test classes run in parallel
> we do reset currently during test run but it hasn't helped, so marking in
pom for sequential seems to be the only resort
I think you are a bit confused about what parallel/sequential means. I
stated this [a few days
ago](https://github.com/apache/hadoop/pull/3341#discussion_r696867103); nothing
has changed in Maven since then.
1. In the parallel test run phase, Maven surefire/failsafe spawns multiple
processes to execute test suite in parallel.
2. Within a process, tests are run sequentially.
The issue is not parallel execution, the issue is state persisting from
previous runs.
The reason things work in sequential & not parallel is the
`<reuseForks>false</reuseForks>` flag. That's all. it has nothing to do with
parallel test runs -not directly. Indirectly it could be due to something like:
the fact tests take longer to run, or that the destination store is updated by
other processes.
Before moving into sequential phase, and so making test runs slower, can we
work out why the `statistics.reset();` calls aren't working. As I keep saying:
it is not failing from parallel tests running.
Meaning it could be one of
* stats references not binding properly
* counters double/under-counting in some circumstances.
* state of remote store affecting results.
* something remaining active in the process
* reset() not working.
I think we need to understand more before committing this -otherwise it's
not so much a fix for the problem as a "we changed this and it went away" kind
of PR.
It probably is just some state left over from the previous test run -but
there is a risk of it being something real.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 644014)
Time Spent: 2h 10m (was: 2h)
> ABFS: Fix transient failures in ITestAbfsStreamStatistics and
> ITestAbfsRestOperationException
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-17873
> URL: https://issues.apache.org/jira/browse/HADOOP-17873
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.3.1
> Reporter: Sumangala Patki
> Assignee: Sumangala Patki
> Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> To address transient failures in the following test classes:
> * ITestAbfsStreamStatistics: Uses a filesystem level instance to record
> read/write statistics, which also tracks these operations in other tests.
> running parallelly. To be marked for sequential run only to avoid transient
> failure
> * ITestAbfsRestOperationException: The use of a static member to track retry
> count causes transient failures when two tests of this class happen to run
> together. Switch to non-static variable for assertions on retry count
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
