Zehao Chen created HDFS-15811:
---------------------------------
Summary: completeFile should log final file size
Key: HDFS-15811
URL: https://issues.apache.org/jira/browse/HDFS-15811
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Zehao Chen
Assignee: Zehao Chen
Jobs, particularly hive queries by non-headless users, can create an excessive
number of files (many hundreds of thousands). A single user's query can
generate a sustained burst of 60-80% of all creates for tens of minutes or more
and impact overall cluster performance. Adding the file size to the logline
allows us to identify excessive tiny or large files.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
