BEGIN:VCALENDAR METHOD:REQUEST PRODID:Microsoft Exchange Server 2010 VERSION:2.0 BEGIN:VTIMEZONE TZID:Eastern Standard Time BEGIN:STANDARD DTSTART:16010101T020000 TZOFFSETFROM:-0400 TZOFFSETTO:-0500 RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=1SU;BYMONTH=11 END:STANDARD BEGIN:DAYLIGHT DTSTART:16010101T020000 TZOFFSETFROM:-0500 TZOFFSETTO:-0400 RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=2SU;BYMONTH=3 END:DAYLIGHT END:VTIMEZONE BEGIN:VEVENT ORGANIZER;CN=Michael O'Brien:MAILTO:[email protected] ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=onap-discu [email protected]:MAILTO:[email protected] DESCRIPTION;LANGUAGE=en-US:Team\,\n We will having another meeting on Th u at 1130 EDT GMT-4 to discuss ELK performance issues – everyone is welc ome\n Notes:\n\n * An idle ONAP deployment pushes 30+ logs/sec (1+ G b/day) into the ELK stack – we are going through optimizing the stack.\n * Logstash was saturating a single VM of a cluster before we put in a temp fix to up the replicaSet from 1 to 3 – there is a change under test /review for a DaemonSet (1 container/vm) – we will discuss this\n * Logstash issue - https://jira.onap.org/browse/LOG-376\n * S3P ELK epic - https://jira.onap.org/browse/LOG-258\n * A root-cause-analysis w ill be attempted both for the southbound filebeat push of logs into logsta sh and the northbound pull from elasticsearch during indexing\n * Th e 30 logs/sec causes periodic vCore peaks of 7\n * The full resource op timization of all of ONAP is being started by Mike E. under https://jira.o nap.org/browse/OOM-927 to start - we will work with these changes\n * T he resource requirements of most of the components in OOM are defaulted/co mmented – we will look at the ELK containers in this session\n * Fine tuning the CPU/RAM only for ELK may be problematic unless we tune all ONA P components together (prioritize) – in this session we will just do the 3 log containers\n * If we have time we will cover off\n * Use o f a load balancer serviceType – we are taking on faith that the current service distributes load properly on the DaemonSet\n * GC heap usage – if it is an issue\n * Elasticsearch shard settings (beyond defa ults)\n * Oscillation behavior under a forced 2g or 2core limit stop /starting container\n * Determine the sweet spot for horizontal clus tering of es and ls\n * Determine the effect of cpu/ram resource lim its on other pods in particular vms\n * Elasticsearch messagebroker usage – if es is overloading ls\n * Future: ElasticSearch as a ser vice for clamp/aai/log\n\nThank you\n/michael and the logging team\n SUMMARY;LANGUAGE=en-US:[OOM][LOG] Performance S3P tuning - part 1 ELK (logs tash/elasticsearch scaling) DTSTART;TZID=Eastern Standard Time:20180524T113000 DTEND;TZID=Eastern Standard Time:20180524T123000 UID:040000008200E00074C5B7101A82E0080000000020B9371CE4F0D301000000000000000 0100000007F8EE14386FDC843955887120EB6BEC2 CLASS:PUBLIC PRIORITY:5 DTSTAMP:20180521T135527Z TRANSP:OPAQUE STATUS:CONFIRMED SEQUENCE:0 LOCATION;LANGUAGE=en-US:http://zoom.us/j/7939937123 X-MICROSOFT-CDO-APPT-SEQUENCE:0 X-MICROSOFT-CDO-OWNERAPPTID:-495626270 X-MICROSOFT-CDO-BUSYSTATUS:TENTATIVE X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY X-MICROSOFT-CDO-ALLDAYEVENT:FALSE X-MICROSOFT-CDO-IMPORTANCE:1 X-MICROSOFT-CDO-INSTTYPE:0 X-MICROSOFT-DISALLOW-COUNTER:FALSE BEGIN:VALARM ACTION:DISPLAY DESCRIPTION:REMINDER TRIGGER;RELATED=START:-PT15M END:VALARM END:VEVENT END:VCALENDAR
_______________________________________________ onap-discuss mailing list [email protected] https://lists.onap.org/mailman/listinfo/onap-discuss
