[
https://issues.apache.org/jira/browse/TEZ-2628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated TEZ-2628:
----------------------------
Attachment: TEZ-2628.003.patch
We ran into a problem with the domain files and HiveServer2. The domain files
were relying on the client JVM shutting down to close the files, but
HiveServer2 never exits and the files were never closing. This caused a pileup
of open files over time.
Here's an updated patch against master that changes the domain file handling so
each domain put writes the data and closes the file. This will not be ideal if
there are many domain puts occurring for an application, but in practice we see
only one domain put per app.
> History logging plugin to write ATS events to HDFS
> --------------------------------------------------
>
> Key: TEZ-2628
> URL: https://issues.apache.org/jira/browse/TEZ-2628
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: TEZ-2628.001.patch, TEZ-2628.002.patch,
> TEZ-2628.003.patch, hive-timeline.json
>
>
> This provides another history logging alternative that conceptually the same
> as the timeline logging service but logs the entities to a file rather than
> posting the events to the timeline server directly. When coupled with the
> timeline store plugin from YARN-3942 it allows the Tez job to be decoupled
> from the timeline server yet the Tez UI can still function properly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
