leerho opened a new issue, #12: URL: https://github.com/apache/datasketches-tck/issues/12
The current workflow pins one commit per source language and updates one language at a time after review. That gives reproducibility, but it doesn't match how cross-language testing has to work in practice: - **Nobody can review ~1000 `.sk` files by hand.** What matters is whether each language's tests pass against the other languages' snapshots. That's an automated question, not a manual review. - **Each language validates its own output first.** A language's generators and its own round-trip tests run in that language's CI before anything reaches the TCK. A snapshot set that passes there is ready to publish. - **Requiring every other language to test one language's new files before its pin can move is an N² gate.** With 5–6 languages updating independently, it wouldn't converge. Proposal: 1. **The TCK publishes the latest self-tested snapshots from every language.** For example, a scheduled (non-required) workflow, or a single command, runs `snapshots update <lang> main` for each language and commits the result, so that TCK `main` always means "latest from every language". 2. **Consumers test against the latest by default.** A language repo asks the TCK for the latest snapshot set and runs its cross-language tests. Failures show exactly which language and sketch disagree, which is the signal we want, as early as possible. 3. **Pins stay available for reproducibility**, e.g. for a release, or for a CI run that must not change underneath us. But they're an option, not the gate. The existing `stability.go` classification remains useful: a summary of the *deterministic* snapshots that changed is a quick, meaningful check. A handful of files changing, like a flag bit, is easy to understand, while the probabilistic files are expected to change on every run. If this direction works for you but you're pressed for time, let us know and we can prepare the PRs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
