[
https://issues.apache.org/jira/browse/HIVE-28989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17955840#comment-17955840
]
Stamatis Zampetakis commented on HIVE-28989:
--------------------------------------------
The best way to implement the proposed check is using the maven-enforcer plugin
and the
[requireFilesSize|https://maven.apache.org/enforcer/enforcer-rules/requireFilesSize.html].
However, currently the rule cannot compute the size for an entire directory so
I raised a [PR|https://github.com/apache/maven-enforcer/pull/368] to support
this enhancement. Once this is a merged we will be able to implement the target
size check rather easily and integrate it in the Hive build.
> Add size threshold for mavens target directory
> ----------------------------------------------
>
> Key: HIVE-28989
> URL: https://issues.apache.org/jira/browse/HIVE-28989
> Project: Hive
> Issue Type: Task
> Reporter: Stamatis Zampetakis
> Assignee: Stamatis Zampetakis
> Priority: Major
>
> The target directory tends to store all kind of information from jar
> artifacts to test logs, reports, and many other things. It is not uncommon,
> that small seemingly innocent changes (e.g., adding a new test, refactoring)
> can make the target directory unreasonably large consuming many GB of disk
> space. When such changes sneak into the project there is a high risk of
> causing CI/CD instability and intermittent failures that may be hard to debug
> and fix. Quite recently we have seen such disk based failures in CI
> (HIVE-28954) that were somehow mitigated by fixing HIVE-28955.
> In order to prevent these disk based problems to go unnoticed, we should
> implement a check that raises an error when the target directory becomes
> unreasonably large. It mostly makes sense to implement this check after the
> tests have run since they are among the main culprits that can increase disk
> usage.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
