maheshrajus opened a new pull request, #519:
URL: https://github.com/apache/tez/pull/519
#### Problem
org.apache.tez.history.TestHistoryParser.testParserWithSuccessfulJob fails
intermittently with:
```
JSONException: A JSONObject text must begin with '{' at character 0 of
at ATSFileParser.readJson(ATSFileParser.java:181)
at ATSFileParser.parseATSZipFile(...)
at ATSFileParser.getDAGData(...)
```
#### Root cause
After the DAG client returns, ATSHistoryLoggingService still has history
events in an async queue that must be
flushed to the timeline server. The test previously called
ATSImportTool.process(...) immediately, so under load the
download could race the timeline write path — **producing a zip whose
entries were empty/whitespace**, which then failed JSON parsing with the
misleading "must begin with {" error.
A fixed Thread.sleep(10000) was already present before the SimpleHistory
parse path (as a workaround for the same class of race), but there was no
equivalent guard for the ATS parse path.
#### Changes
1. ATSFileParser:
- skip empty/whitespace zip entries with a WARN; enrich JSON parse
errors with the offending entry name + payload snippet.
2. TestHistoryParser:
-replaced the unguarded ATS export+parse with a retry loop that only
accepts a DagInfo with ≥2
vertices, tasks, and attempts.
-replaced the fixed Thread.sleep(10000) before SimpleHistory parse with a
poll-until-file-size-stable.
3. New TestATSFileParser:
-New tests added that checks (empty entry skipped, malformed entry names
itself in the error).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
