On 19/02/18 9:23 AM, Jaikiran Pai wrote:
On 19/02/18 1:37 AM, Stefan Bodewig wrote:
On 2018-02-15, Jaikiran Pai wrote:

We have 3 pre-defined formatters all of which are capable of receiving
this streamed sysout/syserr data. Each of these do _not_ hold on to
this sysout/syserr data in-memory during the execution and instead
stream it out a temporary file. Once the execution completes and it's
time to present the result, these formatters stream/read back the
content from the temp file and write it out in a formatted manner to
the target report files.
Sounds expensive.
Indeed. In 2 ways actually:

    1. The creation of these files.
    2. The fact that we write it out to these files (although in a separate thread) and then read it while formatting the result, finally.

It's only applicable for result formatters which want to have the sysout and syserr content sent to them, but I expect that most users will want this enabled anyway. So yes, it's expensive.

The alternate approach is perhaps to have this content maintained in-memory till a fixed size limit (relatively large enough but not too much that it will cause OOM) and only create and stream into temporary files if that size limit is exceeded. That way the temp file creation and writing/reading perhaps can be avoided/controlled in most of the cases.
I just updated the PR to include an additional commit which introduces this idea. The (common internal) formatter class now has a "store" which is used to store the sysout/syserr content. The store is initially backed by a fixed size ByteBuffer and the incoming sysout/syserr content is pushed to that. When/if the size limit is reached, the store is switched to create a new temporary file and write out the already held data and any new incoming data. Once the file is created, the in-memory buffer is destroyed. I've intentionally kept this as a completely internal implementation detail which means that the size limit of this internal buffer isn't configurable. I (somewhat randomly) decided to use 50KB as the size limit for (each) sysout/syserr store but am willing to change it to anything lower (or maybe even higher) that might make sense.

The PR is the same one, that we have been using so far for review here[1] and the specific commit is this one[2]. I'll be squashing all those commits into one before merging, once the review is done.

Thanks Stefan for the review so far, it has been of real help and uncovered some bugs as well as my lack of knowledge of some of the Ant task creation guidelines.

[1] https://github.com/apache/ant/pull/60
[2] https://github.com/apache/ant/pull/60/commits/641b873b9e9db5d78c7cd7953da135fb0b9c097e


To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

Reply via email to