Hi bigtop .... it’s been a while since peters news so figured I’d try to cheer everyone up with some news about my latest big data / kubernetes adventures.
I recently got a new use case for some load testing in k8s networks, and wanted to have an intuitive way to measure overall cluster throughout on unhealthy clusters. So I thought I’d let folks know we’ve been using the BPS transaction queue for it . The name of this new project I’ve been hacking on with a friend of mine, Ben Fenwick. https://github.com/carbonrelay/kapture It launches transactions into a Kafka spout and then reads them into various redis sinks. It’s a really cool way to bring down and load test the network, CPU, and memory on any cluster. For me, im using it to compare network performance in clouds vs bare metal. On this note , id still love to explore the idea of a next generation cloud native style big data stack , heavily focused on kv and messaging , and I think a fork of this project could be the seeds of such a thing.
