[
https://issues.apache.org/jira/browse/YARN-6125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15850732#comment-15850732
]
Andras Piros commented on YARN-6125:
------------------------------------
* {{ExpectedException.none()}} as factory method, and {{ExpectedException}}
general usage description can be found
[*here*|https://github.com/junit-team/junit4/wiki/exception-testing#expectedexception-rule].
Basically if you don't expect any {{Exception}} s to be thrown, you don't call
{{expect()}}, {{expectMessage()}} or {{expectCause()}}. Yes, you can always
test exceptional behavior w/ [*try-catch
idiom*|https://github.com/junit-team/junit4/wiki/exception-testing#trycatch-idiom]
instead of {{ExpectedException}}
* got rid of {{Lists.newLinkedList()}} call
* got rid of that {{final}} variable
Posting the patch addressing those issues but not yet the truncate behavior.
Talked to [~templedf] and [~yufeigu] offline about truncate characteristics,
here is one possible solution:
# when a message is being appended, and it doesn't fit into the buffer, trim
message body to buffer size (minus header prefixes) preserving its head and add
a prefix header telling users it's been partially truncated
# when a message is being appended, and it fits into the buffer:
## past messages are truncated in FIFO order
## for all the ones truncated fully we put a common prefix header stating there
have been messages left out
## for the last one truncated partially we trim the message body and add a
prefix header stating it's been partially truncated
Some example scenarios:
# nothing truncated:
{noformat}
message1 this is a very long message that fits into the buffer
message2 this is a lot shorter than the previour one
message3 the shortest one
{noformat}
# only the very last message has been partially truncated:
{noformat}
message1 this is a very long message that fits into the buffer
message2 this is a lot shorter than the previour one
message3 the shortest one
<MESSAGE TAIL TRUNCATED>message4 this has been one that did not fit totally
into the buffer...
{noformat}
# only the very first message has been partially truncated:
{noformat}
<MESSAGE TAIL TRUNCATED>message1 this has been one that did not fit totally
into the buffer...
message2 this is a very long message that fits into the buffer
message3 this is a lot shorter than the previour one
message4 the shortest one
{noformat}
# the first several ones have been deleted, the previous one has been truncated
partially:
{noformat}
<PREVIOUS MESSAGES DELETED>
<MESSAGE TAIL TRUNCATED>message3 this has been one that did not fit totally
into the buffer...
message4 the shortest one
{noformat}
# all the previous ones have been deleted, plus the last one has been truncated:
{noformat}
<PREVIOUS MESSAGES DELETED>
<MESSAGE TAIL TRUNCATED>message4 this has been one that did not fit totally
into the buffer...
{noformat}
[~jlowe], [~kasha], [~vvasudev] what is your opinion about the proposal?
> The application attempt's diagnostic message should have a maximum size
> -----------------------------------------------------------------------
>
> Key: YARN-6125
> URL: https://issues.apache.org/jira/browse/YARN-6125
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
> Affects Versions: 2.7.0
> Reporter: Daniel Templeton
> Assignee: Andras Piros
> Priority: Critical
> Fix For: 3.0.0-alpha3
>
> Attachments: YARN-6125.000.patch, YARN-6125.001.patch,
> YARN-6125.002.patch
>
>
> We've found through experience that the diagnostic message can grow
> unbounded. I've seen attempts that have diagnostic messages over 1MB. Since
> the message is stored in the state store, it's a bad idea to allow the
> message to grow unbounded. Instead, there should be a property that sets a
> maximum size on the message.
> I suspect that some of the ZK state store issues we've seen in the past were
> due to the size of the diagnostic messages and not to the size of the
> classpath, as is the current prevailing opinion.
> An open question is how best to prune the message once it grows too large.
> Should we
> # truncate the tail,
> # truncate the head,
> # truncate the middle,
> # add another property to make the behavior selectable, or
> # none of the above?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
