Spidey01 commented on issue #8614:
URL: https://github.com/apache/trafficserver/issues/8614#issuecomment-1745232464
This morning, I tried reproducing this with ATS 9.2.1 and it just works as
expected - issue does not reproduce.
```
==> /var/log/trafficserver/access.log <==
{"localtimestamp": "03/Oct/2023:11:16:00 -0400", "client_ip": "10.211.55.2",
"cache_response": "TCP_MISS", "cache_response_code": "200", "request_page":
"http://cache-vm/MHawk/rolling-buffer/itcrowd/index.m3u8";, "server_response":
"200", "bytes": 158, "user_agent": "thundering-herd.sh/horse2", "sessionId":
"01_2329856082", "origin_response_time": 2, "client_response_time": 56,
"ssl_connection": 0, "referrer": "-"}
{"localtimestamp": "03/Oct/2023:11:16:00 -0400", "client_ip": "10.211.55.2",
"cache_response": "TCP_MISS", "cache_response_code": "200", "request_page":
"http://cache-vm/MHawk/rolling-buffer/itcrowd/index.m3u8";, "server_response":
"200", "bytes": 158, "user_agent": "thundering-herd.sh/horse1", "sessionId":
"01_2329856081", "origin_response_time": 1, "client_response_time": 68,
"ssl_connection": 0, "referrer": "-"}
{"localtimestamp": "03/Oct/2023:11:16:00 -0400", "client_ip": "10.211.55.2",
"cache_response": "TCP_HIT", "cache_response_code": "200", "request_page":
"http://cache-vm/MHawk/rolling-buffer/itcrowd/index.m3u8";, "server_response":
"000", "bytes": 158, "user_agent": "thundering-herd.sh/horse4", "sessionId":
"01_2329856084", "origin_response_time": -1, "client_response_time": 145,
"ssl_connection": 0, "referrer": "-"}
{"localtimestamp": "03/Oct/2023:11:16:00 -0400", "client_ip": "10.211.55.2",
"cache_response": "TCP_MEM_HIT", "cache_response_code": "200", "request_page":
"http://cache-vm/MHawk/rolling-buffer/itcrowd/index.m3u8";, "server_response":
"000", "bytes": 158, "user_agent": "thundering-herd.sh/horse3", "sessionId":
"01_2329856083", "origin_response_time": -1, "client_response_time": 194,
"ssl_connection": 0, "referrer": "-"}
```
I'm not sure if this is related to bug fixes between 9.1.3 and 9.2.1, or
changes to other timeout settings. My VM's read-while writer related settings
haven't changed since my previous comment, but my timeout settings relative to
the default have since been simplified.
```
grep timeout records.config
CONFIG proxy.config.http.connect_attempts_timeout INT 1
CONFIG proxy.config.http.keep_alive_no_activity_timeout_in INT 115
CONFIG proxy.config.http.transaction_active_timeout_in INT 40
CONFIG proxy.config.http.transaction_active_timeout_out INT 2
CONFIG proxy.config.http.transaction_no_activity_timeout_out INT 2
CONFIG proxy.config.http.background_fill_active_timeout INT 0
```
For reference, here's an updated copy of thundering-herd.sh, not that it
should make a difference in reproducibility.
```sh
#!/bin/sh
default_rate=""
rate=$default_rate
lead_rate="$rate"
default_maxtime=9999999
maxtime=$default_maxtime
lead_maxtime=$maxtime
output=""
lead_output=$output
stride=0
usage() {
echo "$(basename $0) [-m MAXTIME | -M MAXTIME] [-r RATE | -R RATE] URL
SIZE INTERVAL"
echo
echo " -m MAXTIME Set the curl --max-time for each horse. Default is
${maxtime}."
echo " -M MAXTIME Set the curl --max-time for lead horse (overrides
-m)."
echo " -r RATE Set a curl --limit-rate for each horse. Default is
not add --limit-rate."
echo " -R RATE Set a curl --limit-rate for lead horse (overrides
-r)."
echo " -o OUTPUT Set a curl --output for each horse. The horse
number will be appended to OUTPUT."
echo " -O OUTPUT Set a curl --output for lead horse. The horse
number will be appended to OUTPUT."
echo " -s STRIDE Modify every horse settings by stride."
echo " E.g., -s 3 => every 3rd horse gets -r, -m; rest
get defaults."
echo " Note that -o is always set, because curl treats
binary data to terminal as an abort."
echo
echo "*WARNING* the lead horse in this context is client one. Which may
NOT be the connection that the server chooses as the lead horse due to other
constraints such as thread scheduling, TCP setup, etc"
echo
echo "URL: the URL to hit from each client."
echo "SIZE: herd size, how many clients."
echo "INTERVAL: stagger horses by sleep INTERVAL"
}
while getopts "r:R:m:M:o:O:s:h" arg; do
case "$arg" in
m)
maxtime=$OPTARG
;;
M)
lead_maxtime=$OPTARG
;;
r)
rate=$OPTARG
;;
R)
lead_rate=$OPTARG
;;
o)
output=$OPTARG
;;
O)
lead_output=$OPTARG
;;
s)
stride=$OPTARG
;;
h)
usage
exit
;;
\?)
[ -n "$OPTARG" ] && echo "Invalid option: -$OPTARG"
exit $OPTERR
;;
esac
done
shift $(($OPTIND - 1))
url="$1"
size="$2"
interval="$3"
echo "url: $url size: $size interval: $interval maxtime: $maxtime"
[ -z $url -o -z $size ] && echo "must specify url and size" && usage && exit
i=0
while [ $i -lt $size ]; do
i=$(($i + 1))
echo "start client $i"
target_maxtime=
target_rate=
[ -n "$output" ] && target_output="--output $output"
if [ $stride -eq 0 ]; then
# Apply to every horse.
[ -n "$maxtime" ] && target_maxtime="--max-time $maxtime"
[ -n "$rate" ] && target_rate="--limit-rate $rate"
elif [ $stride -gt 0 ]; then
if [ $(($i % $stride)) -eq 0 ]; then
# Apply every stride'th horse.
echo "$stride 'th horse"
[ -n "$maxtime" ] && target_maxtime="--max-time $maxtime"
[ -n "$rate" ] && target_rate="--limit-rate $rate"
else
target_maxtime=""
target_rate=""
fi
fi
[ -n "$lead_maxtime" -a $i -eq 1 ] && target_maxtime="--max-time
$lead_maxtime"
[ -n "$lead_rate" -a $i -eq 1 ] && target_rate="--limit-rate $lead_rate"
[ -n "$lead_output" -a $i -eq 1 ] && target_output="--output
$lead_output"
user_agent="-A thundering-herd.sh/horse$i"
echo "horse $i target_maxtime: $target_maxtime target_rate: $target_rate
stride: $stride user_agent: $user_agent"
# continue
case $(uname -s) in
Darwin)
# macOS uses BSD style script.
script /tmp/curl.$i.out curl $target_output $target_maxtime
$target_rate $user_agent -v "$url" -L &
;;
Linux)
# Assume util-linux style script.
script -c "curl $target_output $target_maxtime $target_rate
$user_agent -v '$url' -L" /tmp/curl.$i.out &
;;
esac
[ -n "$interval" ] && sleep $interval
done
# Be nice to terminal.
wait
echo
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
