andygrove commented on issue #949: URL: https://github.com/apache/datafusion-ballista/issues/949#issuecomment-5004172406
Running TPC-H against a multi-node deployment is now documented in the Contributors Guide: https://datafusion.apache.org/ballista/contributors-guide/benchmarking.html It covers the reference cluster shape and the reasoning behind the parts that are easy to get wrong (one executor per node, node-local data, headroom between the memory pool and the container limit), self-contained scheduler/executor/`tpch` invocations, and the current results. The page is deliberately tooling-agnostic rather than shipping Kubernetes manifests, so it does not include the Dockerfile and pod YAML this issue proposed. That is a conscious trade: manifests in-tree need maintaining, and the commands work the same whether they are run under Kubernetes, docker-compose, or bare metal. Closing as covered. The YAML in the description has aged out regardless -- it references an image that no longer exists and a `--concurrency` flag that has since become `--partitions`. If there is still appetite for runnable k8s manifests in-tree, that is worth a fresh issue describing what they should do. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
