Thanks for the quick response, Wu Sheng. Env detail: The Java services are deployed into K8S pod with 18 instances: CPU 3 cores, Memory: 6G The OAP services are deployed into K8S pod with 3 instances: CPU 2 cores, Memory: 2G
The performance pressure test for Java services is about 6500 cps. I grep single POD skywaking logs for "abandoned" and 700,000 + records were found. Thanks & Best Regards Xiaochao Zhang(James) DI SW CAS MP EMK DO-CHN No.7, Xixin Avenue, Chengdu High-Tech Zone Chengdu, China 611731 Cellphone: +86 13980787820 Email: [email protected] -----Original Message----- From: Sheng Wu <[email protected]> Sent: Wednesday, January 8, 2020 3:49 PM To: dev <[email protected]> Subject: Re: Java agent buffer setting to avoid trace segment abandon Hi Zhang Welcome to join the dev ml. How much payload do you put in the tests? Does your backend and storage are powerful enough? I would prefer you could share more information about your test env. Sheng Wu 吴晟 Twitter, wusheng1108 Zhang, James <[email protected]> 于2020年1月8日周三 下午3:46写道： > Dear Skywalking Dev, > I found a lot of "trace segment has been abandoned, cause by buffer is > full" logs in my Java services with Skywalking 6.6.0 agent enabled. > DEBUG 2020-01-06 20:43:17:699 http-nio-0.0.0.0-9090-exec-154 > TraceSegmentServiceClient : One trace segment has been abandoned, > cause by buffer is full. > > And some "xxx trace segments have been abandoned, cause by no > available channel" logs were found also. > 2020-01-06 21:37:53:716 DataCarrier.DEFAULT.Consumser.0.Thread > TraceSegmentServiceClient : 237 trace segments have been abandoned, > cause by no available channel. > > I checked the source code & documentation found that the default > buffer setting is 5(channel_size)*300(buffer_size) and it seems that > this default setting is not enough for productive environment of heavy load > system. > > To avoid the trace segment abandon, is that OK to just increase the > buffer setting (e.g. to 10* 3000) ? How to estimate the memory(in MB) > for the buffer setting so that I can evaluate the memory footprint for > segments buffer? > > Thanks & Best Regards > > Xiaochao Zhang(James) > DI SW CAS MP EMK DO-CHN > No.7, Xixin Avenue, Chengdu High-Tech Zone Chengdu, China 611731 > Email: [email protected] <mailto:[email protected]> > >
