Hi, @bmahler I think this should be a expected behavior when open the webui of no-leanding masters, because their metrics about frameworks and tasks would not updated.
On Sep 29, 2016 4:12 AM, "Benjamin Mahler" <[email protected]> wrote: > Thanks for reporting this Rodrick, do you see any errors in your browser's > console? > > On Tue, Sep 27, 2016 at 4:29 AM, Rodrick Brown <[email protected]> > wrote: > >> >> On Sep 27, 2016, at 3:43 AM, haosdent <[email protected]> wrote: >> >> Hi, @Rodrick >> >> >"master/frameworks_connected": 0, >> >> Is it because the master you open no the leading master? >> >> >> >> Sorry you’re right the leader has switched over, so it seems the stats >> are their but the UI isn’t able to render this information in the Task >> pane. >> >> $ curl -s http://leader.mesos:5050/metrics/snapshot | jq >> { >> "master/messages_status_update": 11461, >> "master/messages_unregister_slave": 0, >> "master/messages_reconcile_tasks": 5660, >> "master/tasks_lost": 59, >> "master/messages_decline_offers": 783792, >> "master/invalid_status_update_acknowledgements": 0, >> "system/load_5min": 0.23, >> "master/tasks_failed": 2202, >> "master/messages_launch_tasks": 4051, >> "master/messages_resource_request": 0, >> "master/messages_status_update_acknowledgement": 8398, >> "master/slaves_connected": 16, >> "master/event_queue_http_requests": 0, >> "master/messages_deactivate_framework": 189, >> "master/messages_reregister_slave": 33, >> "master/messages_executor_to_framework": 0, >> "registrar/log/recovered": 1, >> "master/slave_removals/reason_registered": 10, >> "master/messages_suppress_offers": 101, >> "master/uptime_secs": 62643.405314816, >> "allocator/mesos/resources/disk/total": 2547171, >> "registrar/queued_operations": 0, >> "system/load_1min": 0.08, >> "master/slaves_disconnected": 0, >> "master/frameworks_active": 13, >> "master/elected": 1, >> "master/valid_executor_to_framework_messages": 0, >> "allocator/mesos/resources/mem/total": 890692, >> "master/valid_status_updates": 8409, >> "system/load_15min": 0.33, >> "allocator/event_queue_dispatches": 1, >> "master/slaves_active": 16, >> "registrar/state_store_ms/p9999": 77.609414272, >> "registrar/state_store_ms/p95": 33.68032, >> "master/frameworks_connected": 13, >> "allocator/mesos/roles/sparkr/shares/dominant": 0.202054133190822, >> "master/messages_exited_executor": 3, >> "master/cpus_total": 220, >> "master/mem_revocable_percent": 0, >> "master/slave_registrations": 25, >> "allocator/mesos/roles/*/shares/dominant": 0.353636363636364, >> "allocator/mesos/event_queue_dispatches": 1, >> "master/tasks_running": 92, >> "master/slaves_inactive": 0, >> "master/messages_register_framework": 764, >> "master/event_queue_dispatches": 17, >> "master/messages_update_slave": 34, >> "master/mem_revocable_total": 0, >> "master/messages_reregister_framework": 204, >> "master/dropped_messages": 12, >> "master/tasks_staging": 0, >> "allocator/mesos/resources/cpus/offered_or_allocated": 112.6, >> "master/tasks_error": 0, >> "master/invalid_framework_to_executor_messages": 0, >> "master/invalid_executor_to_framework_messages": 0, >> "master/slave_shutdowns_scheduled": 15, >> "master/slave_removals/reason_unregistered": 0, >> "master/frameworks_inactive": 1, >> "master/tasks_finished": 1819, >> "master/frameworks_disconnected": 1, >> "master/disk_revocable_used": 0, >> "master/messages_authenticate": 0, >> "registrar/state_store_ms/max": 77.662208, >> "master/event_queue_messages": 0, >> "master/slave_shutdowns_canceled": 0, >> "master/messages_kill_task": 369, >> "master/slave_reregistrations": 16, >> "allocator/mesos/allocation_run_ms/p999": 4.60756198399999, >> "registrar/state_fetch_ms": 8.230144, >> "master/gpus_used": 0, >> "allocator/mesos/allocation_run_ms/min": 0.338176, >> "master/disk_total": 2547171, >> "master/disk_used": 67770, >> "master/valid_status_update_acknowledgements": 8398, >> "master/mem_revocable_used": 0, >> "master/task_lost/source_master/reason_slave_removed": 40, >> "master/gpus_percent": 0, >> "master/slave_removals/reason_unhealthy": 15, >> "master/task_failed/source_slave/reason_container_launch_failed": 1250, >> "master/slave_removals": 25, >> "master/slave_shutdowns_completed": 15, >> "allocator/mesos/allocation_runs": 65597, >> "master/mem_used": 370025, >> "master/invalid_status_updates": 3052, >> "master/recovery_slave_removals": 0, >> "allocator/mesos/allocation_run_ms/p99": 4.10023424, >> "master/task_lost/source_master/reason_slave_disconnected": 12, >> "master/task_failed/source_slave/reason_container_limitation_memory": >> 59, >> "master/messages_framework_to_executor": 0, >> "master/task_killed/source_master/reason_framework_removed": 953, >> "master/task_lost/source_slave/reason_reconciliation": 7, >> "master/disk_revocable_total": 0, >> "master/gpus_total": 0, >> "master/cpus_percent": 0.511818181818182, >> "registrar/state_store_ms/p50": 8.625536, >> "allocator/mesos/offer_filters/roles/*/active": 79, >> "allocator/mesos/allocation_run_ms": 2.353152, >> "registrar/state_store_ms/p90": 21.558784, >> "allocator/mesos/allocation_run_ms/count": 1000, >> "allocator/mesos/offer_filters/roles/sparkr/active": 37, >> "allocator/mesos/allocation_run_ms/max": 5.19296, >> "allocator/mesos/allocation_run_ms/p50": 2.515968, >> "master/messages_register_slave": 64, >> "allocator/mesos/allocation_run_ms/p90": 2.9021184, >> "master/messages_unregister_framework": 729, >> "allocator/mesos/allocation_run_ms/p95": 3.0233472, >> "system/cpus_total": 8, >> "registrar/state_store_ms/p999": 77.13427072, >> "master/mem_percent": 0.415435414262169, >> "allocator/mesos/allocation_run_ms/p9999": 5.13442019840004, >> "master/cpus_revocable_total": 0, >> "master/gpus_revocable_total": 0, >> "master/cpus_revocable_percent": 0, >> "master/gpus_revocable_percent": 0, >> "master/disk_percent": 0.0266059875838725, >> "master/disk_revocable_percent": 0, >> "master/gpus_revocable_used": 0, >> "master/task_failed/source_slave/reason_executor_terminated": 55, >> "master/valid_framework_to_executor_messages": 0, >> "master/cpus_used": 112.6, >> "master/tasks_killed": 1311, >> "registrar/state_store_ms/count": 66, >> "master/cpus_revocable_used": 0, >> "master/tasks_killing": 0, >> "system/mem_total_bytes": 15769341952, >> "master/messages_revive_offers": 2313, >> "allocator/mesos/resources/cpus/total": 220, >> "allocator/mesos/resources/disk/offered_or_allocated": 67770, >> "system/mem_free_bytes": 8169160704, >> "master/mem_total": 890692, >> "registrar/state_store_ms": 10.951936, >> "master/outstanding_offers": 0, >> "registrar/state_store_ms/p99": 72.3828352, >> "registrar/registry_size_bytes": 4393, >> "master/tasks_starting": 0, >> "allocator/mesos/resources/mem/offered_or_allocated": 370025, >> "registrar/state_store_ms/min": 6.721024 >> } >> >> >> On Tue, Sep 27, 2016 at 1:26 PM, Rodrick Brown <[email protected]> >> wrote: >> >>> >>> >>> On Sep 26, 2016, at 11:54 PM, haosdent <[email protected]> wrote: >>> >>> Hi, @Rodrick What is the response when you open ` >>> http://${MASTER_IP}:${MASTER_PORT}/metrics/snapshot`? >>> <http://$%7Bmaster_ip%7D:$%7Bmaster_port%7D/metrics/snapshot%60?> >>> >>> >>> $ curl -s http://master:5050/metrics/snapshot |jq >>> { >>> "master/slave_shutdowns_scheduled": 0, >>> "master/invalid_executor_to_framework_messages": 0, >>> "master/mem_used": 0, >>> "allocator/mesos/allocation_runs": 40793, >>> "master/slave_shutdowns_completed": 0, >>> "master/invalid_status_updates": 0, >>> "master/messages_authenticate": 0, >>> "master/frameworks_disconnected": 0, >>> "master/disk_revocable_used": 0, >>> "master/messages_exited_executor": 0, >>> "master/messages_status_update": 0, >>> "master/messages_unregister_slave": 0, >>> "master/messages_framework_to_executor": 0, >>> "master/messages_reconcile_tasks": 0, >>> "master/tasks_lost": 0, >>> "master/messages_decline_offers": 0, >>> "allocator/mesos/allocation_run_ms/p99": 0.01587456, >>> "master/recovery_slave_removals": 0, >>> "allocator/mesos/allocation_run_ms/max": 0.028928, >>> "master/cpus_total": 0, >>> "master/messages_register_slave": 0, >>> "allocator/mesos/allocation_run_ms/p90": 0.013056, >>> "master/tasks_running": 0, >>> "allocator/mesos/allocation_run_ms/p999": 0.027905024, >>> "master/slave_reregistrations": 0, >>> "master/cpus_percent": 0, >>> "allocator/mesos/resources/mem/offered_or_allocated": 0, >>> "master/messages_register_framework": 0, >>> "allocator/mesos/allocation_run_ms": 0.011008, >>> "allocator/mesos/event_queue_dispatches": 0, >>> "master/tasks_staging": 0, >>> "master/slave_removals/reason_unregistered": 0, >>> "allocator/mesos/allocation_run_ms/count": 1000, >>> "master/event_queue_http_requests": 0, >>> "master/slave_removals": 0, >>> "master/gpus_used": 0, >>> "allocator/mesos/allocation_run_ms/p50": 0.012032, >>> "master/dropped_messages": 0, >>> "allocator/mesos/allocation_run_ms/min": 0.00384, >>> "master/disk_total": 0, >>> "allocator/mesos/resources/disk/offered_or_allocated": 0, >>> "system/mem_free_bytes": 13478334464, >>> "master/slave_removals/reason_unhealthy": 0, >>> "master/gpus_revocable_total": 0, >>> "master/cpus_revocable_total": 0, >>> "allocator/mesos/allocation_run_ms/p9999": 0.0288257024000001, >>> "system/cpus_total": 8, >>> "master/mem_percent": 0, >>> "master/gpus_percent": 0, >>> "master/mem_revocable_used": 0, >>> "master/valid_status_update_acknowledgements": 0, >>> "master/disk_used": 0, >>> "master/messages_unregister_framework": 0, >>> "allocator/mesos/allocation_run_ms/p95": 0.013056, >>> "master/messages_resource_request": 0, >>> "master/slaves_inactive": 0, >>> "master/messages_update_slave": 0, >>> "master/event_queue_dispatches": 3, >>> "master/event_queue_messages": 0, >>> "allocator/mesos/resources/cpus/offered_or_allocated": 0, >>> "master/cpus_revocable_used": 0, >>> "system/mem_total_bytes": 15769341952, >>> "master/tasks_killing": 0, >>> "allocator/mesos/resources/cpus/total": 0, >>> "master/messages_revive_offers": 0, >>> "master/gpus_revocable_percent": 0, >>> "master/disk_percent": 0, >>> "master/messages_kill_task": 0, >>> "master/slave_shutdowns_canceled": 0, >>> "master/gpus_revocable_used": 0, >>> "master/disk_revocable_percent": 0, >>> "master/frameworks_connected": 0, >>> "master/slave_registrations": 0, >>> "master/cpus_revocable_percent": 0, >>> "master/mem_revocable_percent": 0, >>> "master/disk_revocable_total": 0, >>> "master/gpus_total": 0, >>> "master/tasks_finished": 0, >>> "master/frameworks_inactive": 0, >>> "master/outstanding_offers": 0, >>> "master/valid_framework_to_executor_messages": 0, >>> "master/cpus_used": 0, >>> "master/tasks_killed": 0, >>> "master/invalid_framework_to_executor_messages": 0, >>> "master/tasks_error": 0, >>> "master/messages_reregister_framework": 0, >>> "master/mem_revocable_total": 0, >>> "master/tasks_starting": 0, >>> "master/mem_total": 0, >>> "master/slaves_active": 0, >>> "allocator/event_queue_dispatches": 1, >>> "system/load_15min": 0, >>> "master/valid_status_updates": 0, >>> "allocator/mesos/resources/mem/total": 0, >>> "master/valid_executor_to_framework_messages": 0, >>> "master/elected": 0, >>> "master/frameworks_active": 0, >>> "master/slaves_disconnected": 0, >>> "system/load_1min": 0, >>> "registrar/queued_operations": 0, >>> "allocator/mesos/resources/disk/total": 0, >>> "master/uptime_secs": 40841.21039104, >>> "master/messages_suppress_offers": 0, >>> "master/slave_removals/reason_registered": 0, >>> "registrar/log/recovered": 1, >>> "master/messages_executor_to_framework": 0, >>> "master/messages_reregister_slave": 0, >>> "master/messages_deactivate_framework": 0, >>> "master/slaves_connected": 0, >>> "master/messages_status_update_acknowledgement": 0, >>> "master/messages_launch_tasks": 0, >>> "system/load_5min": 0, >>> "master/tasks_failed": 0, >>> "master/invalid_status_update_acknowledgements": 0 >>> } >>> >>> >>> >>> On Tue, Sep 27, 2016 at 11:43 AM, Rodrick Brown < >>> [email protected]> wrote: >>> >>>> I just upgraded our cluster from 0.28.2 to 1.0.1 and notice the Web UI >>>> task details are no longer updating. I'm trying to see if this issue is >>>> isolated to my setup? or broken across the board I was not able to find >>>> anything about this bug for 1.0.1 >>>> >>>> >>>> Tasks >>>> Staging 0 >>>> Starting 0 >>>> Running 0 >>>> Killing 0 >>>> Finished 0 >>>> Killed 0 >>>> Failed 0 >>>> Lost 0 >>>> Orphan 21 >>>> >>>> >>>> >>>> -- >>>> >>>> [image: Orchard Platform] <http://www.orchardplatform.com/> >>>> >>>> *Rodrick Brown */ *Lead SRE - DevOps* >>>> >>>> 9174456839 / [email protected] >>>> >>>> Orchard Platform >>>> 101 5th Avenue, 4th Floor, New York, NY >>>> >>>> *NOTICE TO RECIPIENTS*: This communication is confidential and >>>> intended for the use of the addressee only. If you are not an intended >>>> recipient of this communication, please delete it immediately and >>>> notify the sender by return email. Unauthorized reading, dissemination, >>>> distribution or copying of this communication is prohibited. This >>>> communication does not constitute an offer to sell or a solicitation >>>> of an indication of interest to purchase any loan, security or any other >>>> financial product or instrument, nor is it an offer to sell or a >>>> solicitation of an indication of interest to purchase any products or >>>> services to any persons who are prohibited from receiving such information >>>> under applicable law. The contents of this communication may not >>>> be accurate or complete and are subject to change without notice. As such, >>>> Orchard App, Inc. (including its subsidiaries and affiliates, "Orchard") >>>> makes no representation regarding the accuracy or completeness of the >>>> information contained herein. The intended recipient is advised to consult >>>> its own professional advisors, including those specializing in legal, tax >>>> and accounting matters. Orchard does not provide legal, tax or >>>> accounting advice. >>>> >>> >>> >>> >>> -- >>> Best Regards, >>> Haosdent Huang >>> >>> >>> >>> *NOTICE TO RECIPIENTS*: This communication is confidential and intended >>> for the use of the addressee only. If you are not an intended recipient of >>> this communication, please delete it immediately and notify the sender >>> by return email. Unauthorized reading, dissemination, distribution or >>> copying of this communication is prohibited. This communication does not >>> constitute >>> an offer to sell or a solicitation of an indication of interest to purchase >>> any loan, security or any other financial product or instrument, nor is it >>> an offer to sell or a solicitation of an indication of interest to purchase >>> any products or services to any persons who are prohibited from receiving >>> such information under applicable law. The contents of this communication >>> may not be accurate or complete and are subject to change without notice. >>> As such, Orchard App, Inc. (including its subsidiaries and affiliates, >>> "Orchard") makes no representation regarding the accuracy or >>> completeness of the information contained herein. The intended recipient is >>> advised to consult its own professional advisors, including those >>> specializing in legal, tax and accounting matters. Orchard does not >>> provide legal, tax or accounting advice. >>> >> >> >> >> -- >> Best Regards, >> Haosdent Huang >> >> >> >> *NOTICE TO RECIPIENTS*: This communication is confidential and intended >> for the use of the addressee only. If you are not an intended recipient of >> this communication, please delete it immediately and notify the sender >> by return email. Unauthorized reading, dissemination, distribution or >> copying of this communication is prohibited. This communication does not >> constitute >> an offer to sell or a solicitation of an indication of interest to purchase >> any loan, security or any other financial product or instrument, nor is it >> an offer to sell or a solicitation of an indication of interest to purchase >> any products or services to any persons who are prohibited from receiving >> such information under applicable law. The contents of this communication >> may not be accurate or complete and are subject to change without notice. >> As such, Orchard App, Inc. (including its subsidiaries and affiliates, >> "Orchard") makes no representation regarding the accuracy or >> completeness of the information contained herein. The intended recipient is >> advised to consult its own professional advisors, including those >> specializing in legal, tax and accounting matters. Orchard does not >> provide legal, tax or accounting advice. >> > >
