[jira] [Updated] (YARN-10418) Add css attribute "word-break" to Diagnostic messages in webUI of RM
[ https://issues.apache.org/jira/browse/YARN-10418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dapeng Sun updated YARN-10418: -- Attachment: after.png > Add css attribute "word-break" to Diagnostic messages in webUI of RM > > > Key: YARN-10418 > URL: https://issues.apache.org/jira/browse/YARN-10418 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Dapeng Sun >Assignee: Dapeng Sun >Priority: Trivial > Attachments: YARN-10418.001.patch, after.png, before.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10418) Add css attribute "word-break" to Diagnostic messages in webUI of RM
[ https://issues.apache.org/jira/browse/YARN-10418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dapeng Sun updated YARN-10418: -- Attachment: before.png > Add css attribute "word-break" to Diagnostic messages in webUI of RM > > > Key: YARN-10418 > URL: https://issues.apache.org/jira/browse/YARN-10418 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Dapeng Sun >Assignee: Dapeng Sun >Priority: Trivial > Attachments: YARN-10418.001.patch, before.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10418) Add css attribute "word-break" to Diagnostic messages in webUI of RM
[ https://issues.apache.org/jira/browse/YARN-10418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dapeng Sun updated YARN-10418: -- Attachment: YARN-10418.001.patch > Add css attribute "word-break" to Diagnostic messages in webUI of RM > > > Key: YARN-10418 > URL: https://issues.apache.org/jira/browse/YARN-10418 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Dapeng Sun >Assignee: Dapeng Sun >Priority: Trivial > Attachments: YARN-10418.001.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10418) Add css attribute "word-break" to Diagnostic messages in webUI of RM
[ https://issues.apache.org/jira/browse/YARN-10418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dapeng Sun updated YARN-10418: -- Summary: Add css attribute "word-break" to Diagnostic messages in webUI of RM (was: Diagnostic messages in RM webUI is better to add "word-break" css attribute) > Add css attribute "word-break" to Diagnostic messages in webUI of RM > > > Key: YARN-10418 > URL: https://issues.apache.org/jira/browse/YARN-10418 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Dapeng Sun >Assignee: Dapeng Sun >Priority: Trivial > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10418) Diagnostic messages in RM webUI is better to add "word-break" css attribute
Dapeng Sun created YARN-10418: - Summary: Diagnostic messages in RM webUI is better to add "word-break" css attribute Key: YARN-10418 URL: https://issues.apache.org/jira/browse/YARN-10418 Project: Hadoop YARN Issue Type: Improvement Reporter: Dapeng Sun Assignee: Dapeng Sun -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9198) Corrupted state from a previous version can still cause RM to fail with NPE on FairScheduler
[ https://issues.apache.org/jira/browse/YARN-9198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743618#comment-16743618 ] Dapeng Sun commented on YARN-9198: -- {quote} Not restoring an application is irreversible. There is no way to get that application back. If that would be an application that had been running for some time (like days) processing petabytes of data not restoring the application could be far more costly than some extra down time. {quote} Yes, in this scenario, we should not skip the error application. How about adding an config, the key likes "xxx.resourcemanager.fair-scheduler.skip-error-apps", so that users could choose from the behaviors: "Stoping RM and Recover the error App" or "Skip Error and Continue Starting RM". The option could be false by default, when meet the exception, the log would show the id(s) of error applications, user could make the decision to "fix" or "skip" base on the logs. > Corrupted state from a previous version can still cause RM to fail with NPE > on FairScheduler > > > Key: YARN-9198 > URL: https://issues.apache.org/jira/browse/YARN-9198 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler, resourcemanager >Affects Versions: 3.1.0, 2.8.5 >Reporter: Dapeng Sun >Assignee: Dapeng Sun >Priority: Major > Attachments: YARN-9198.001.patch > > > Previously, RM may fail with NPE due to YARN-4347,YARN-4000. After these > fixes, FairScheduler still has the same potential issue. > > 201x-xx-xx xx:xx:xx,xxx ERROR resourcemanager.ResourceManager > (ResourceManager.java:serviceStart) - Failed to load/recover state > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplicationAttempt(FairScheduler.java) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9198) Corrupted state from a previous version can still cause RM to fail with NPE on FairScheduler
[ https://issues.apache.org/jira/browse/YARN-9198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16742132#comment-16742132 ] Dapeng Sun edited comment on YARN-9198 at 1/14/19 2:58 PM: --- Most of your opinions are reasonable for me, it is better to find and fix the underlying issue around FairScheduler. like the queue issue, config or other reasons which break restoring of app state. But in product mode, recovering RM is more important at most time. If RM can't work rightly, all the works would be blocked, it would be much worse than an application can't be restore. For users who care about why the application is not restored, they could also check the reason at log and dig into it. Do you have any ideas? was (Author: dapengsun): Most of your options are reasonable for me, it is better to find and fix the underlying issue around FairScheduler. like the queue issue, config or other reasons which break restoring of app state. But in product mode, recovering RM is more important at most time. If RM can't work rightly, all the works would be blocked, it would be much worse than an application can't be restore. For users who care about why the application is not restored, they could also check the reason at log and dig into it. Do you have any ideas? > Corrupted state from a previous version can still cause RM to fail with NPE > on FairScheduler > > > Key: YARN-9198 > URL: https://issues.apache.org/jira/browse/YARN-9198 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler, resourcemanager >Affects Versions: 3.1.0, 2.8.5 >Reporter: Dapeng Sun >Assignee: Dapeng Sun >Priority: Major > Attachments: YARN-9198.001.patch > > > Previously, RM may fail with NPE due to YARN-4347,YARN-4000. After these > fixes, FairScheduler still has the same potential issue. > > 201x-xx-xx xx:xx:xx,xxx ERROR resourcemanager.ResourceManager > (ResourceManager.java:serviceStart) - Failed to load/recover state > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplicationAttempt(FairScheduler.java) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9198) Corrupted state from a previous version can still cause RM to fail with NPE on FairScheduler
[ https://issues.apache.org/jira/browse/YARN-9198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16742132#comment-16742132 ] Dapeng Sun commented on YARN-9198: -- Most of your options are reasonable for me, it is better to find and fix the underlying issue around FairScheduler. like the queue issue, config or other reasons which break restoring of app state. But in product mode, recovering RM is more important at most time. If RM can't work rightly, all the works would be blocked, it would be much worse than an application can't be restore. For users who care about why the application is not restored, they could also check the reason at log and dig into it. Do you have any ideas? > Corrupted state from a previous version can still cause RM to fail with NPE > on FairScheduler > > > Key: YARN-9198 > URL: https://issues.apache.org/jira/browse/YARN-9198 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler, resourcemanager >Affects Versions: 3.1.0, 2.8.5 >Reporter: Dapeng Sun >Assignee: Dapeng Sun >Priority: Major > Attachments: YARN-9198.001.patch > > > Previously, RM may fail with NPE due to YARN-4347,YARN-4000. After these > fixes, FairScheduler still has the same potential issue. > > 201x-xx-xx xx:xx:xx,xxx ERROR resourcemanager.ResourceManager > (ResourceManager.java:serviceStart) - Failed to load/recover state > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplicationAttempt(FairScheduler.java) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9198) Corrupted state from a previous version can still cause RM to fail with NPE on FairScheduler
[ https://issues.apache.org/jira/browse/YARN-9198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16742049#comment-16742049 ] Dapeng Sun commented on YARN-9198: -- Hi [~wilfreds], thank you for your comments :) The exception reported here is also thrown by [FairScheduler.java#L494|https://github.com/apache/hadoop/blob/55066cc53dc22b68f9ca55a0029741d6c846be0a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java#L494] as YARN-7913 mentioned. It happened on RM failover, and RM can not change the state from standby to active due to NPE before I reformat the state. I just pick up how Capacity Scheduler ([CapacityScheduler.java#L875|https://github.com/apache/hadoop/blob/55066cc53dc22b68f9ca55a0029741d6c846be0a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java#L875]) handle this kind of exception for quick fix. > Corrupted state from a previous version can still cause RM to fail with NPE > on FairScheduler > > > Key: YARN-9198 > URL: https://issues.apache.org/jira/browse/YARN-9198 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler, resourcemanager >Affects Versions: 3.1.0, 2.8.5 >Reporter: Dapeng Sun >Assignee: Dapeng Sun >Priority: Major > Attachments: YARN-9198.001.patch > > > Previously, RM may fail with NPE due to YARN-4347,YARN-4000. After these > fixes, FairScheduler still has the same potential issue. > > 201x-xx-xx xx:xx:xx,xxx ERROR resourcemanager.ResourceManager > (ResourceManager.java:serviceStart) - Failed to load/recover state > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplicationAttempt(FairScheduler.java) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9198) Corrupted state from a previous version can still cause RM to fail with NPE on FairScheduler
[ https://issues.apache.org/jira/browse/YARN-9198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16741887#comment-16741887 ] Dapeng Sun edited comment on YARN-9198 at 1/14/19 9:38 AM: --- Attached a simple patch to fix it. was (Author: dapengsun): Attach a simple patch to fix it. > Corrupted state from a previous version can still cause RM to fail with NPE > on FairScheduler > > > Key: YARN-9198 > URL: https://issues.apache.org/jira/browse/YARN-9198 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler, resourcemanager >Affects Versions: 3.1.0, 2.8.5 >Reporter: Dapeng Sun >Assignee: Dapeng Sun >Priority: Major > Attachments: YARN-9198.001.patch > > > Previously, RM may fail with NPE due to YARN-4347,YARN-4000. After these > fixes, FairScheduler still has the same potential issue. > > 201x-xx-xx xx:xx:xx,xxx ERROR resourcemanager.ResourceManager > (ResourceManager.java:serviceStart) - Failed to load/recover state > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplicationAttempt(FairScheduler.java) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9198) Corrupted state from a previous version can still cause RM to fail with NPE on FairScheduler
[ https://issues.apache.org/jira/browse/YARN-9198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16741887#comment-16741887 ] Dapeng Sun commented on YARN-9198: -- Attach a simple patch to fix it. > Corrupted state from a previous version can still cause RM to fail with NPE > on FairScheduler > > > Key: YARN-9198 > URL: https://issues.apache.org/jira/browse/YARN-9198 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler, resourcemanager >Affects Versions: 3.1.0, 2.8.5 >Reporter: Dapeng Sun >Assignee: Dapeng Sun >Priority: Major > Attachments: YARN-9198.001.patch > > > Previously, RM may fail with NPE due to YARN-4347,YARN-4000. After these > fixes, FairScheduler still has the same potential issue. > > 201x-xx-xx xx:xx:xx,xxx ERROR resourcemanager.ResourceManager > (ResourceManager.java:serviceStart) - Failed to load/recover state > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplicationAttempt(FairScheduler.java) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9198) Corrupted state from a previous version can still cause RM to fail with NPE on FairScheduler
[ https://issues.apache.org/jira/browse/YARN-9198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dapeng Sun updated YARN-9198: - Attachment: YARN-9198.001.patch > Corrupted state from a previous version can still cause RM to fail with NPE > on FairScheduler > > > Key: YARN-9198 > URL: https://issues.apache.org/jira/browse/YARN-9198 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler, resourcemanager >Affects Versions: 3.1.0, 2.8.5 >Reporter: Dapeng Sun >Assignee: Dapeng Sun >Priority: Major > Attachments: YARN-9198.001.patch > > > Previously, RM may fail with NPE due to YARN-4347,YARN-4000. After these > fixes, FairScheduler still has the same potential issue. > > 201x-xx-xx xx:xx:xx,xxx ERROR resourcemanager.ResourceManager > (ResourceManager.java:serviceStart) - Failed to load/recover state > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplicationAttempt(FairScheduler.java) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9198) Corrupted state from a previous version can still cause RM to fail with NPE on FairScheduler
Dapeng Sun created YARN-9198: Summary: Corrupted state from a previous version can still cause RM to fail with NPE on FairScheduler Key: YARN-9198 URL: https://issues.apache.org/jira/browse/YARN-9198 Project: Hadoop YARN Issue Type: Bug Components: fairscheduler, resourcemanager Affects Versions: 2.8.5, 3.1.0 Reporter: Dapeng Sun Assignee: Dapeng Sun Previously, RM may fail with NPE due to YARN-4347,YARN-4000. After these fixes, FairScheduler still has the same potential issue. 201x-xx-xx xx:xx:xx,xxx ERROR resourcemanager.ResourceManager (ResourceManager.java:serviceStart) - Failed to load/recover state java.lang.NullPointerException at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplicationAttempt(FairScheduler.java) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-646) Some issues in Fair Scheduler's document
[ https://issues.apache.org/jira/browse/YARN-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dapeng Sun updated YARN-646: Attachment: YARN-646.patch > Some issues in Fair Scheduler's document > > > Key: YARN-646 > URL: https://issues.apache.org/jira/browse/YARN-646 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 2.0.4-alpha >Reporter: Dapeng Sun > Fix For: 2.0.5-beta > > Attachments: YARN-646.patch, YARN-646.patch > > > Issues are found in the doc page for Fair Scheduler > http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/FairScheduler.html: > 1.In the section “Configuration”, It contains two properties named > “yarn.scheduler.fair.minimum-allocation-mb”, the second one should be > “yarn.scheduler.fair.maximum-allocation-mb” > 2.In the section “Allocation file format”, the document tells “ The format > contains three types of elements”, but it lists four types of elements > following that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-646) Some issues in Fair Scheduler's document
[ https://issues.apache.org/jira/browse/YARN-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13649609#comment-13649609 ] Dapeng Sun commented on YARN-646: - Uploaded a simple patch that fixes these. > Some issues in Fair Scheduler's document > > > Key: YARN-646 > URL: https://issues.apache.org/jira/browse/YARN-646 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 2.0.4-alpha >Reporter: Dapeng Sun > Fix For: 2.0.5-beta > > Attachments: YARN-646.patch > > > Issues are found in the doc page for Fair Scheduler > http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/FairScheduler.html: > 1.In the section “Configuration”, It contains two properties named > “yarn.scheduler.fair.minimum-allocation-mb”, the second one should be > “yarn.scheduler.fair.maximum-allocation-mb” > 2.In the section “Allocation file format”, the document tells “ The format > contains three types of elements”, but it lists four types of elements > following that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-646) Some issues in Fair Scheduler's document
[ https://issues.apache.org/jira/browse/YARN-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dapeng Sun updated YARN-646: Attachment: YARN-646.patch > Some issues in Fair Scheduler's document > > > Key: YARN-646 > URL: https://issues.apache.org/jira/browse/YARN-646 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Affects Versions: 2.0.4-alpha >Reporter: Dapeng Sun > Fix For: 2.0.5-beta > > Attachments: YARN-646.patch > > > Issues are found in the doc page for Fair Scheduler > http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/FairScheduler.html: > 1.In the section “Configuration”, It contains two properties named > “yarn.scheduler.fair.minimum-allocation-mb”, the second one should be > “yarn.scheduler.fair.maximum-allocation-mb” > 2.In the section “Allocation file format”, the document tells “ The format > contains three types of elements”, but it lists four types of elements > following that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (YARN-646) Some issues in Fair Scheduler's document
Dapeng Sun created YARN-646: --- Summary: Some issues in Fair Scheduler's document Key: YARN-646 URL: https://issues.apache.org/jira/browse/YARN-646 Project: Hadoop YARN Issue Type: Bug Components: documentation Affects Versions: 2.0.4-alpha Reporter: Dapeng Sun Fix For: 2.0.5-beta Issues are found in the doc page for Fair Scheduler http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/FairScheduler.html: 1.In the section “Configuration”, It contains two properties named “yarn.scheduler.fair.minimum-allocation-mb”, the second one should be “yarn.scheduler.fair.maximum-allocation-mb” 2.In the section “Allocation file format”, the document tells “ The format contains three types of elements”, but it lists four types of elements following that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (YARN-518) Fair Scheduler's document link could be added to the hadoop 2.x main doc page
Dapeng Sun created YARN-518: --- Summary: Fair Scheduler's document link could be added to the hadoop 2.x main doc page Key: YARN-518 URL: https://issues.apache.org/jira/browse/YARN-518 Project: Hadoop YARN Issue Type: Improvement Components: documentation Affects Versions: 2.0.3-alpha Reporter: Dapeng Sun Currently the doc page for Fair Scheduler looks good and it’s here, http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/FairScheduler.html. It would be better to add the document link to the YARN section in the Hadoop 2.x main doc page, so that users can easily find the doc to experimentally try Fair Scheduler as Capacity Scheduler. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira