Interesting figures, especially that the failures seem to persist even on different major releases. I initially thought the failures existed because I am testing ATS in a VM. So, yesterday I installed the same OS and ATS release, compiled the source as provided, on a bare HP server with 2x240 GB SSD in RAID1 for the OS and 2 x 1TB each in its own volume (RAID0) for the cache storage... However, I still have failures (see below)
"proxy.process.version.server.short": "8.0.7", "proxy.process.version.server.long": "Apache Traffic Server - traffic_server - 8.0.7 - (build # 042720 on Apr 27 2020 at 20:11:28)", "proxy.process.version.server.build_number": "042720", "proxy.process.version.server.build_time": "20:11:28", "proxy.process.version.server.build_date": "Apr 27 2020", "proxy.process.version.server.build_person": "root", "proxy.process.http.background_fill_current_count": "4", "proxy.process.http.current_client_connections": "640", "proxy.process.http.current_active_client_connections": "122", "proxy.process.http.websocket.current_active_client_connections": "0", "proxy.process.http.current_client_transactions": "261", "proxy.process.http.current_server_transactions": "184", "proxy.process.http.current_parent_proxy_connections": "0", "proxy.process.http.current_server_connections": "805", "proxy.process.http.current_cache_connections": "155", "proxy.process.cache.volume_0.bytes_used": "65715343360", "proxy.process.cache.volume_0.bytes_total": "1917932191744", "proxy.process.cache.volume_0.ram_cache.total_bytes": "25769803776", "proxy.process.cache.volume_0.ram_cache.bytes_used": "770314880", "proxy.process.cache.volume_0.ram_cache.hits": "541749", "proxy.process.cache.volume_0.ram_cache.misses": "115283", "proxy.process.cache.volume_0.pread_count": "0", "proxy.process.cache.volume_0.percent_full": "3", "proxy.process.cache.volume_0.lookup.active": "0", "proxy.process.cache.volume_0.lookup.success": "0", "proxy.process.cache.volume_0.lookup.failure": "0", "proxy.process.cache.volume_0.read.active": "4", "proxy.process.cache.volume_0.read.success": "525727","proxy.process.cache.volume_0.read.failure": "1290873", "proxy.process.cache.volume_0.write.active": "153", "proxy.process.cache.volume_0.write.success": "222118","proxy.process.cache.volume_0.write.failure": "42813", "proxy.process.cache.volume_0.write.backlog.failure": "0", "proxy.process.cache.volume_0.update.active": "1", "proxy.process.cache.volume_0.update.success": "33634","proxy.process.cache.volume_0.update.failure": "533", "proxy.process.cache.volume_0.remove.active": "0", "proxy.process.cache.volume_0.remove.success": "0", "proxy.process.cache.volume_0.remove.failure": "0", "proxy.process.cache.volume_0.evacuate.active": "0", "proxy.process.cache.volume_0.evacuate.success": "0", "proxy.process.cache.volume_0.evacuate.failure": "0", "proxy.process.cache.volume_0.scan.active": "0", "proxy.process.cache.volume_0.scan.success": "0", "proxy.process.cache.volume_0.scan.failure": "0", "proxy.process.cache.volume_0.direntries.total": "239453928", "proxy.process.cache.volume_0.direntries.used": "274305", "proxy.process.cache.volume_0.directory_collision": "0", "proxy.process.cache.volume_0.frags_per_doc.1": "228249", "proxy.process.cache.volume_0.frags_per_doc.2": "0", "proxy.process.cache.volume_0.frags_per_doc.3+": "10884", "proxy.process.cache.volume_0.read_busy.success": "105","proxy.process.cache.volume_0.read_busy.failure": "52600", "proxy.process.cache.volume_0.write_bytes_stat": "0", "proxy.process.cache.volume_0.vector_marshals": "0", "proxy.process.cache.volume_0.hdr_marshals": "0", "proxy.process.cache.volume_0.hdr_marshal_bytes": "0", "proxy.process.cache.volume_0.gc_bytes_evacuated": "0", "proxy.process.cache.volume_0.gc_frags_evacuated": "0", "proxy.process.cache.volume_0.wrap_count": "0", "proxy.process.cache.volume_0.sync.count": "252", "proxy.process.cache.volume_0.sync.bytes": "302145806336", "proxy.process.cache.volume_0.sync.time": "72939367496484", "proxy.process.cache.volume_0.span.errors.read": "0", "proxy.process.cache.volume_0.span.errors.write": "0", "proxy.process.cache.volume_0.span.failing": "0", "proxy.process.cache.volume_0.span.offline": "0", "proxy.process.cache.volume_0.span.online": "0", "server": "8.0.7" On Tue, Apr 28, 2020 at 6:58 PM Bryan Call <[email protected]> wrote: > Here are some numbers from our production servers, from a couple different > groups. We are running a heavily modified version of 7.1.2 > > One of our production server for our CDN: > proxy.process.cache.read.success 6037258 > proxy.process.cache.read.failure 13845799 > > Here is another one from another group: > proxy.process.cache.read.success 5575072 > proxy.process.cache.read.failure 26784750 > > I talked to another company and they were seeing about a 3% failure rate > on their cache and they are using 8.0.7. I created an issue on this: > https://github.com/apache/trafficserver/issues/6713 > > -Bryan > > > On Apr 26, 2020, at 4:32 PM, edd! <[email protected]> wrote: > > > > Hi, > > > > New to ATS, I compiled 8.0.7 from source yesterday on CentOS 7 > > # source /opt/rh/devtoolset-7/enable > > # ./configure --enable-experimental-plugins > > # make && make install > > Testing it as a transparent forward proxy serving ~500 users with http > caching enabled. I have first tried raw cache storage then volume file but > in both cases I got so many read and few write failures on a 100 GB SSD > partition. > > > > "proxy.process.cache.volume_0.bytes_used": "3323351040", > > "proxy.process.cache.volume_0.bytes_total": "106167836672", > > "proxy.process.cache.volume_0.ram_cache.total_bytes": "12884901888", > > "proxy.process.cache.volume_0.ram_cache.bytes_used": "6062080", > > "proxy.process.cache.volume_0.ram_cache.hits": "4916", > > "proxy.process.cache.volume_0.ram_cache.misses": "1411", > > "proxy.process.cache.volume_0.pread_count": "0", > > "proxy.process.cache.volume_0.percent_full": "3", > > "proxy.process.cache.volume_0.lookup.active": "0", > > "proxy.process.cache.volume_0.lookup.success": "0", > > "proxy.process.cache.volume_0.lookup.failure": "0", > > "proxy.process.cache.volume_0.read.active": "1", > > "proxy.process.cache.volume_0.read.success": "5566", > > "proxy.process.cache.volume_0.read.failure": "22084", > > "proxy.process.cache.volume_0.write.active": "8", > > "proxy.process.cache.volume_0.write.success": "5918", > > "proxy.process.cache.volume_0.write.failure": "568", > > "proxy.process.cache.volume_0.write.backlog.failure": "272", > > "proxy.process.cache.volume_0.update.active": "1", > > "proxy.process.cache.volume_0.update.success": "306", > > "proxy.process.cache.volume_0.update.failure": "4", > > "proxy.process.cache.volume_0.remove.active": "0", > > "proxy.process.cache.volume_0.remove.success": "0", > > "proxy.process.cache.volume_0.remove.failure": "0", > > "proxy.process.cache.volume_0.evacuate.active": "0", > > "proxy.process.cache.volume_0.evacuate.success": "0", > > "proxy.process.cache.volume_0.evacuate.failure": "0", > > "proxy.process.cache.volume_0.scan.active": "0", > > "proxy.process.cache.volume_0.scan.success": "0", > > "proxy.process.cache.volume_0.scan.failure": "0", > > "proxy.process.cache.volume_0.direntries.total": "13255088", > > "proxy.process.cache.volume_0.direntries.used": "9230", > > "proxy.process.cache.volume_0.directory_collision": "0", > > "proxy.process.cache.volume_0.frags_per_doc.1": "5372", > > "proxy.process.cache.volume_0.frags_per_doc.2": "0", > > "proxy.process.cache.volume_0.frags_per_doc.3+": "625", > > "proxy.process.cache.volume_0.read_busy.success": "4", > > "proxy.process.cache.volume_0.read_busy.failure": "351", > > > > Disk read/write tests: > > hdparm -t /dev/sdb1 > > /dev/sdb1: > > Timing buffered disk reads: 196 MB in 3.24 seconds = 60.54 MB/sec > > > > hdparm -T /dev/sdb1 > > /dev/sdb1: > > Timing cached reads: 11662 MB in 1.99 seconds = 5863.27 MB/sec > > > > dd if=/dev/zero of=/cache/test1.img bs=1G count=1 oflag=dsync > > 1+0 records in > > 1+0 records out > > 1073741824 bytes (1.1 GB) copied, 67.6976 s, 15.9 MB/s > > > > dd if=/dev/zero of=/cache/test2.img bs=512 count=1000 oflag=dsync > > 1000+0 records in > > 1000+0 records out > > 512000 bytes (512 kB) copied, 0.374173 s, 1.4 MB/s > > > > Please help, > > > > Thank you, > > Eddi > >
