keith-turner commented on PR #6010:
URL: https://github.com/apache/accumulo/pull/6010#issuecomment-3629710171
I have been testing this using Jaeger. The following json was pulled from
Jaeger for a single span and it shows some of the new tags in this PR. It
shows a scan batch read 560,364 bytes from DFS, which turned into 1,109,757
uncompressed bytes read and finally returned 54 bytes to the user after
filtering. It shows that 78,343 key values were read before any filtering was
done by the scan and after filtering only 3 key values were returned. The data
cache was turned off for the table, the stats show that 12 rfile data blocks
were read bypassing the cache. The index cache was turned on, for the 1 rfile
index block read it shows up as a cache hit.
```json
"tags": [
{
"key": "accumulo.bytes.read",
"type": "int64",
"value": 1109757
},
{
"key": "accumulo.bytes.read.file",
"type": "int64",
"value": 560364
},
{
"key": "accumulo.bytes.returned",
"type": "int64",
"value": 54
},
{
"key": "accumulo.cache.data.bypasses",
"type": "int64",
"value": 12
},
{
"key": "accumulo.cache.data.hits",
"type": "int64",
"value": 0
},
{
"key": "accumulo.cache.data.misses",
"type": "int64",
"value": 0
},
{
"key": "accumulo.cache.index.bypasses",
"type": "int64",
"value": 0
},
{
"key": "accumulo.cache.index.hits",
"type": "int64",
"value": 1
},
{
"key": "accumulo.cache.index.misses",
"type": "int64",
"value": 0
},
{
"key": "accumulo.entries.read",
"type": "int64",
"value": 78343
},
{
"key": "accumulo.entries.returned",
"type": "int64",
"value": 3
},
{
"key": "accumulo.executor",
"type": "string",
"value": "default"
},
{
"key": "accumulo.extent",
"type": "string",
"value": "1;04fe;04de"
},
{
"key": "accumulo.seeks",
"type": "int64",
"value": 1
},
{
"key": "accumulo.server",
"type": "string",
"value": "localhost:9997"
},
{
"key": "accumulo.table.id",
"type": "string",
"value": "1"
},
{
"key": "otel.scope.name",
"type": "string",
"value": "org.apache.accumulo"
},
{
"key": "otel.scope.version",
"type": "string",
"value": "2.1.5-SNAPSHOT"
},
{
"key": "span.kind",
"type": "string",
"value": "internal"
},
{
"key": "thread.id",
"type": "int64",
"value": 1215
},
{
"key": "thread.name",
"type": "string",
"value": "Client: localhost:58832 User: root
Start: 1765238204805 Tablet: 1;04fe;04de"
}
],
```
Wanted to see if its possible to pull the data from all spans from Jaeger
and sum them up. This allows seeing the stats for a single scan as it hits
many servers across the cluster. Wrote the following script to do this.
```bash
#!/bin/bash
YSERVER=$1
TRACE_ID=$2
trace_tmp="$(mktemp -t trace.XXXXXXXX.json)"
curl -s http://$YSERVER:16686/api/traces/$TRACE_ID?prettyPrint=false >
$trace_tmp
echo "Sum of all traces"
cat $trace_tmp | jq -r '.data[0].spans[] | .tags | .[] | [.key,.type,.value]
| @csv' | grep int64 | grep -v thread.id | awk -F ',' '{a[$1] += $3} END{for (i
in a) print "\t" i, a[i]}' | sort
```
Ran the script against a traced a scan that filtered data on 128 tablets
across 4 tservers. That worked well and the 333 entries returned is how many
key values that scan actually returned, so that was a good test. The json from
Jaeger had at least 128 spans, like the example above, that needed to be rolled
up.
```
$ ./sum_scan_traces.sh localhost c22f939eb5dc1ba294b6aeae7941ebb5
Sum of all traces
"accumulo.bytes.read" 162803236
"accumulo.bytes.read.file" 82245238
"accumulo.bytes.returned" 5972
"accumulo.cache.data.bypasses" 1694
"accumulo.cache.data.hits" 0
"accumulo.cache.data.misses" 0
"accumulo.cache.index.bypasses" 0
"accumulo.cache.index.hits" 191
"accumulo.cache.index.misses" 0
"accumulo.entries.read" 9988264
"accumulo.entries.returned" 333
"accumulo.seeks" 128
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
