[ https://issues.apache.org/jira/browse/YARN-6125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15850732#comment-15850732 ]
Andras Piros commented on YARN-6125: ------------------------------------ * {{ExpectedException.none()}} as factory method, and {{ExpectedException}} general usage description can be found [*here*|https://github.com/junit-team/junit4/wiki/exception-testing#expectedexception-rule]. Basically if you don't expect any {{Exception}} s to be thrown, you don't call {{expect()}}, {{expectMessage()}} or {{expectCause()}}. Yes, you can always test exceptional behavior w/ [*try-catch idiom*|https://github.com/junit-team/junit4/wiki/exception-testing#trycatch-idiom] instead of {{ExpectedException}} * got rid of {{Lists.newLinkedList()}} call * got rid of that {{final}} variable Posting the patch addressing those issues but not yet the truncate behavior. Talked to [~templedf] and [~yufeigu] offline about truncate characteristics, here is one possible solution: # when a message is being appended, and it doesn't fit into the buffer, trim message body to buffer size (minus header prefixes) preserving its head and add a prefix header telling users it's been partially truncated # when a message is being appended, and it fits into the buffer: ## past messages are truncated in FIFO order ## for all the ones truncated fully we put a common prefix header stating there have been messages left out ## for the last one truncated partially we trim the message body and add a prefix header stating it's been partially truncated Some example scenarios: # nothing truncated: {noformat} message1 this is a very long message that fits into the buffer message2 this is a lot shorter than the previour one message3 the shortest one {noformat} # only the very last message has been partially truncated: {noformat} message1 this is a very long message that fits into the buffer message2 this is a lot shorter than the previour one message3 the shortest one <MESSAGE TAIL TRUNCATED>message4 this has been one that did not fit totally into the buffer... {noformat} # only the very first message has been partially truncated: {noformat} <MESSAGE TAIL TRUNCATED>message1 this has been one that did not fit totally into the buffer... message2 this is a very long message that fits into the buffer message3 this is a lot shorter than the previour one message4 the shortest one {noformat} # the first several ones have been deleted, the previous one has been truncated partially: {noformat} <PREVIOUS MESSAGES DELETED> <MESSAGE TAIL TRUNCATED>message3 this has been one that did not fit totally into the buffer... message4 the shortest one {noformat} # all the previous ones have been deleted, plus the last one has been truncated: {noformat} <PREVIOUS MESSAGES DELETED> <MESSAGE TAIL TRUNCATED>message4 this has been one that did not fit totally into the buffer... {noformat} [~jlowe], [~kasha], [~vvasudev] what is your opinion about the proposal? > The application attempt's diagnostic message should have a maximum size > ----------------------------------------------------------------------- > > Key: YARN-6125 > URL: https://issues.apache.org/jira/browse/YARN-6125 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager > Affects Versions: 2.7.0 > Reporter: Daniel Templeton > Assignee: Andras Piros > Priority: Critical > Fix For: 3.0.0-alpha3 > > Attachments: YARN-6125.000.patch, YARN-6125.001.patch, > YARN-6125.002.patch > > > We've found through experience that the diagnostic message can grow > unbounded. I've seen attempts that have diagnostic messages over 1MB. Since > the message is stored in the state store, it's a bad idea to allow the > message to grow unbounded. Instead, there should be a property that sets a > maximum size on the message. > I suspect that some of the ZK state store issues we've seen in the past were > due to the size of the diagnostic messages and not to the size of the > classpath, as is the current prevailing opinion. > An open question is how best to prune the message once it grows too large. > Should we > # truncate the tail, > # truncate the head, > # truncate the middle, > # add another property to make the behavior selectable, or > # none of the above? -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org