400Ping commented on code in PR #1483:
URL: https://github.com/apache/mahout/pull/1483#discussion_r4261252847
##########
testing/qdp/test_benchmark_api.py:
##########
@@ -113,3 +113,51 @@ def test_qdp_benchmark_device_id_propagated():
)
assert hasattr(result, "vectors_per_sec")
assert result.vectors_per_sec >= 0
+
+
+def _qdp_module():
+ """The compiled extension, typed as a plain module so ty accepts its
members."""
+ from qumat_qdp._backend import get_qdp
+
+ qdp = get_qdp()
+ assert qdp is not None
+ return qdp
+
+
+@requires_qdp
[email protected](
+ "encoding", ["amplitude", "angle", "basis", "iqp", "iqp-z", "phase"]
+)
+def test_encoding_supports_f32_matches_parity_grid(encoding):
+ """``_qdp.encoding_supports_f32`` agrees with the f32 kernels the parity
grid checks."""
+ from .test_parity import F32_DEVICE_ENCODINGS
Review Comment:
Importing a constant from another test module is a bit fragile (collection
order, `-p no:cacheprovider` setups). Could `F32_DEVICE_ENCODINGS` live in
`qdp_test_utils.py` so both files import it from there?
##########
qdp/qdp-python/benchmark/baseline.py:
##########
@@ -0,0 +1,209 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Capture and compare a performance baseline for every encoding.
+
+Numbers are machine-specific, so the JSON is meant to live outside the repo
+(a CI artifact or a scratch directory), not be committed.
+
+ # Before an engine refactor:
+ python baseline.py capture --out /tmp/qdp-baseline.json
+
+ # After:
+ python baseline.py compare /tmp/qdp-baseline.json --tolerance 0.05
+
+``compare`` exits non-zero when any encoding's throughput drops by more than
+``--tolerance`` (a fraction) or its latency rises by more than the same
+fraction. Each cell is measured ``--repeats`` times and the best run is kept,
+which removes most of the noise from a shared machine.
+
+An ``f32`` cell is only recorded for encodings with a native float32 pipeline
+path. The pipeline silently measures float64 for the others, so recording
+those under an ``f32`` key would make a future real f32 implementation compare
+against a float64 baseline. Skipped cells are listed in the JSON under
+``"skipped"`` and are ignored by ``compare``.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import platform
+import subprocess
+import sys
+import time
+from typing import Any
+
+from qumat_qdp import QdpBenchmark
+from qumat_qdp._backend import get_qdp
Review Comment:
Heads-up on an ordering conflict: #1505 deletes `qumat_qdp._backend`
(`get_qdp` moves to `_native.py`), so this import and the one in
`test_benchmark_api.py:120` break whichever of the two PRs lands second. Since
`baseline.py` already raises when the extension is missing, the simplest option
that avoids depending on either private module is `import _qdp` directly, the
same way the tests in `testing/qdp/` do. Same applies to `_qdp_module()` in
`test_benchmark_api.py`.
##########
testing/qdp/test_parity.py:
##########
@@ -0,0 +1,202 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Cross-backend parity grid.
+
+``qumat_qdp.torch_ref`` is the specification for every encoding. This module
+runs the native Rust+CUDA engine over the full grid of
+
+ encoding x engine precision x input shape x input location
+
+and checks each cell against the reference. A new encoder is complete when it
+has a reference function and every cell here passes. A refactor of the engine
+or bindings is safe when this file is green before and after.
+"""
+
+from __future__ import annotations
+
+import math
+
+import numpy as np
+import pytest
+import torch
+
+from .qdp_test_utils import requires_qdp
+
+torch_ref = pytest.importorskip("qumat_qdp.torch_ref")
+
+NUM_QUBITS = 4
+BATCH = 5
+
+ENCODINGS = ("amplitude", "angle", "basis", "iqp", "iqp-z", "phase")
+PRECISIONS = ("float64", "float32")
+SHAPES = ("single", "batch")
+LOCATIONS = ("numpy", "torch_cpu", "cuda_f64", "cuda_f32")
+
+# Encodings with a device-side float32 kernel today. Others reject f32 CUDA
+# input, which is covered by ``test_f32_cuda_rejected_without_kernel``.
+F32_DEVICE_ENCODINGS = ("amplitude", "angle", "basis")
+
+# Encodings the current engine cannot read from a CUDA tensor at all. Cells
+# for these are expected failures until the single generic encode path lands;
+# once a listed encoding succeeds the cell fails with a message asking for the
+# entry to be removed, so the list cannot go stale.
+NO_DEVICE_PATH_ENCODINGS = ("phase",)
+
+# A cell's tolerance follows the narrowest precision on its path: the engine's
+# output precision, or float32 when the input arrives as a float32 CUDA tensor
+# and is encoded by the float32 kernel before any widening.
+TOL: dict[str, dict[str, float]] = {
+ "float64": {"atol": 1e-10, "rtol": 1e-9},
+ "float32": {"atol": 2e-5, "rtol": 2e-5},
+}
+
+
+def sample_dim(encoding: str, n: int) -> int:
+ if encoding == "amplitude":
+ return 1 << n
+ if encoding == "basis":
+ return 1
+ if encoding == "iqp":
+ return n + n * (n - 1) // 2
+ return n # angle, phase, iqp-z
+
+
+def make_samples(encoding: str, n: int, batch: int) -> np.ndarray:
+ """Deterministic float64 inputs of shape (batch, sample_dim)."""
+ rng = np.random.default_rng(1234 + len(encoding))
+ dim = sample_dim(encoding, n)
+ if encoding == "amplitude":
+ return rng.uniform(0.1, 1.0, size=(batch, dim))
+ if encoding == "basis":
+ return rng.integers(0, 1 << n, size=(batch, dim)).astype(np.float64)
+ return rng.uniform(0.0, 2.0 * math.pi, size=(batch, dim))
+
+
+def reference(encoding: str, data_2d: np.ndarray, n: int) -> torch.Tensor:
+ x = torch.from_numpy(data_2d).to(torch.float64)
+ return torch_ref.encode(x, n, encoding)
+
+
+def to_input(encoding: str, data_2d: np.ndarray, shape: str, location: str):
+ """Materialise the reference data as the requested input object."""
+ arr = data_2d if shape == "batch" else data_2d[0]
+ if location == "numpy":
+ return np.ascontiguousarray(arr)
+ t = torch.from_numpy(np.ascontiguousarray(arr))
+ if location == "torch_cpu":
+ return t
+ if location == "cuda_f64":
+ # Basis indices on the device are int64 by API contract; every other
+ # encoding takes float64.
+ return (t.to(torch.int64) if encoding == "basis" else t).cuda()
+ if location == "cuda_f32":
+ return t.to(torch.float32).cuda()
+ raise AssertionError(location)
+
+
+def engine_output(qt) -> torch.Tensor:
+ out = torch.from_dlpack(qt)
+ assert out.is_cuda
+ return out
+
+
+def cell_id(p):
+ return str(p)
+
+
+@requires_qdp
[email protected]
[email protected]("location", LOCATIONS, ids=cell_id)
[email protected]("shape", SHAPES, ids=cell_id)
[email protected]("precision", PRECISIONS, ids=cell_id)
[email protected]("encoding", ENCODINGS, ids=cell_id)
+def test_parity_grid(encoding, precision, shape, location):
+ from _qdp import QdpEngine
+
+ if location == "cuda_f32" and encoding not in F32_DEVICE_ENCODINGS:
+ pytest.skip(f"{encoding} has no float32 device kernel")
+
+ engine = QdpEngine(0, precision)
+ data = make_samples(encoding, NUM_QUBITS, BATCH)
+ # The engine always returns a (samples, 2**n) tensor, so a single sample
+ # is compared against the first row of the batch reference.
+ expected = reference(encoding, data, NUM_QUBITS)
+ if shape == "single":
+ expected = expected[:1]
+
+ inp = to_input(encoding, data, shape, location)
+ if location == "cuda_f64" and encoding in NO_DEVICE_PATH_ENCODINGS:
+ try:
+ engine.encode(inp, NUM_QUBITS, encoding)
+ except RuntimeError:
+ pytest.xfail(f"{encoding} has no device-pointer path yet")
+ pytest.fail(
+ f"{encoding} now encodes from a CUDA tensor; "
+ "remove it from NO_DEVICE_PATH_ENCODINGS"
+ )
Review Comment:
pytest has this built in. `pytest.mark.xfail(strict=True,
raises=RuntimeError, reason="phase has no device-pointer path")` gives the same
“remove me when it passes” behaviour, and `raises=` narrows it so an unrelated
error does not count as the known gap. Can be attached via `pytest.param(...,
marks=...)` or in `pytest_collection_modifyitems`.
##########
qdp/qdp-python/benchmark/baseline.py:
##########
@@ -0,0 +1,209 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Capture and compare a performance baseline for every encoding.
+
+Numbers are machine-specific, so the JSON is meant to live outside the repo
+(a CI artifact or a scratch directory), not be committed.
+
+ # Before an engine refactor:
+ python baseline.py capture --out /tmp/qdp-baseline.json
+
+ # After:
+ python baseline.py compare /tmp/qdp-baseline.json --tolerance 0.05
+
+``compare`` exits non-zero when any encoding's throughput drops by more than
+``--tolerance`` (a fraction) or its latency rises by more than the same
+fraction. Each cell is measured ``--repeats`` times and the best run is kept,
+which removes most of the noise from a shared machine.
+
+An ``f32`` cell is only recorded for encodings with a native float32 pipeline
+path. The pipeline silently measures float64 for the others, so recording
+those under an ``f32`` key would make a future real f32 implementation compare
+against a float64 baseline. Skipped cells are listed in the JSON under
+``"skipped"`` and are ignored by ``compare``.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import platform
+import subprocess
+import sys
+import time
+from typing import Any
+
+from qumat_qdp import QdpBenchmark
+from qumat_qdp._backend import get_qdp
+
+ENCODINGS = ("amplitude", "angle", "basis", "iqp", "iqp-z", "phase")
+DTYPES = ("f64", "f32")
+
+
+def supported_cells() -> tuple[list[str], list[str]]:
+ """Split the encoding x dtype grid into measurable and skipped cell keys.
+
+ A cell is skipped when the native pipeline has no float32 path for the
+ encoding and would silently measure float64 instead.
+ """
+ qdp = get_qdp()
+ if qdp is None:
+ raise RuntimeError("The _qdp extension is required to capture a
baseline")
+ cells: list[str] = []
+ skipped: list[str] = []
+ for encoding in ENCODINGS:
+ for dtype in DTYPES:
+ key = f"{encoding}/{dtype}"
+ if dtype == "f32" and not qdp.encoding_supports_f32(encoding):
+ skipped.append(key)
+ else:
+ cells.append(key)
+ return cells, skipped
+
+
+def _gpu_name(device_id: int) -> str:
+ try:
+ out = subprocess.run(
+ [
+ "nvidia-smi",
+ f"--id={device_id}",
+ "--query-gpu=name",
+ "--format=csv,noheader",
+ ],
+ capture_output=True,
+ text=True,
+ check=True,
+ )
+ return out.stdout.strip()
+ except Exception: # informational only
+ return "unknown"
+
+
+def measure(args: argparse.Namespace) -> dict[str, Any]:
+ keys, skipped = supported_cells()
+ cells: dict[str, dict[str, float]] = {}
+ for key in keys:
+ encoding, dtype = key.split("/")
+ bench = (
+ QdpBenchmark(device_id=args.device)
+ .qubits(args.qubits)
+ .encoding(encoding)
+ .batches(args.batches, args.batch_size)
+ .warmup(args.warmup)
+ .dtype(dtype)
+ )
+ best_tp = 0.0
+ best_lat = float("inf")
+ for _ in range(args.repeats):
+ tp = bench.run_throughput()
+ lat = bench.run_latency()
+ best_tp = max(best_tp, tp.vectors_per_sec)
+ best_lat = min(best_lat, lat.latency_ms_per_vector)
+ cells[key] = {"vectors_per_sec": best_tp, "latency_ms_per_vector":
best_lat}
+ print(
+ f"{key:<16} {best_tp:>14,.0f} vec/s {best_lat:>10.5f} ms/vec",
+ flush=True,
+ )
+ for key in skipped:
+ print(f"{key:<16} {'skipped':>14} no native f32 path (would measure
f64)")
+ return {
+ "config": {
+ "qubits": args.qubits,
+ "batches": args.batches,
+ "batch_size": args.batch_size,
+ "warmup": args.warmup,
+ "repeats": args.repeats,
+ },
+ "host": platform.node(),
+ "gpu": _gpu_name(args.device),
+ "captured_at": time.strftime("%Y-%m-%dT%H:%M:%S"),
Review Comment:
`run_pipeline_baseline.py` records the git commit; worth doing the same here
(`git rev-parse HEAD, best effort`) so a `compare` failure can be traced to a
build. Also suggest keeping the raw per-repeat values alongside the best.
##########
qdp/qdp-python/benchmark/baseline.py:
##########
@@ -0,0 +1,209 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Capture and compare a performance baseline for every encoding.
+
+Numbers are machine-specific, so the JSON is meant to live outside the repo
+(a CI artifact or a scratch directory), not be committed.
+
+ # Before an engine refactor:
+ python baseline.py capture --out /tmp/qdp-baseline.json
+
+ # After:
+ python baseline.py compare /tmp/qdp-baseline.json --tolerance 0.05
+
+``compare`` exits non-zero when any encoding's throughput drops by more than
+``--tolerance`` (a fraction) or its latency rises by more than the same
+fraction. Each cell is measured ``--repeats`` times and the best run is kept,
+which removes most of the noise from a shared machine.
+
+An ``f32`` cell is only recorded for encodings with a native float32 pipeline
+path. The pipeline silently measures float64 for the others, so recording
+those under an ``f32`` key would make a future real f32 implementation compare
+against a float64 baseline. Skipped cells are listed in the JSON under
+``"skipped"`` and are ignored by ``compare``.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import platform
+import subprocess
+import sys
+import time
+from typing import Any
+
+from qumat_qdp import QdpBenchmark
+from qumat_qdp._backend import get_qdp
+
+ENCODINGS = ("amplitude", "angle", "basis", "iqp", "iqp-z", "phase")
+DTYPES = ("f64", "f32")
+
+
+def supported_cells() -> tuple[list[str], list[str]]:
+ """Split the encoding x dtype grid into measurable and skipped cell keys.
+
+ A cell is skipped when the native pipeline has no float32 path for the
+ encoding and would silently measure float64 instead.
+ """
+ qdp = get_qdp()
+ if qdp is None:
+ raise RuntimeError("The _qdp extension is required to capture a
baseline")
+ cells: list[str] = []
+ skipped: list[str] = []
+ for encoding in ENCODINGS:
+ for dtype in DTYPES:
+ key = f"{encoding}/{dtype}"
+ if dtype == "f32" and not qdp.encoding_supports_f32(encoding):
+ skipped.append(key)
+ else:
+ cells.append(key)
+ return cells, skipped
+
+
+def _gpu_name(device_id: int) -> str:
+ try:
+ out = subprocess.run(
+ [
+ "nvidia-smi",
+ f"--id={device_id}",
+ "--query-gpu=name",
+ "--format=csv,noheader",
+ ],
+ capture_output=True,
+ text=True,
+ check=True,
+ )
+ return out.stdout.strip()
+ except Exception: # informational only
+ return "unknown"
+
+
+def measure(args: argparse.Namespace) -> dict[str, Any]:
+ keys, skipped = supported_cells()
+ cells: dict[str, dict[str, float]] = {}
+ for key in keys:
+ encoding, dtype = key.split("/")
+ bench = (
+ QdpBenchmark(device_id=args.device)
+ .qubits(args.qubits)
+ .encoding(encoding)
+ .batches(args.batches, args.batch_size)
+ .warmup(args.warmup)
+ .dtype(dtype)
+ )
+ best_tp = 0.0
+ best_lat = float("inf")
+ for _ in range(args.repeats):
Review Comment:
Best-of-N is a sound choice for a noisy shared machine. But the amplitude
throughput numbers in
[apache/mahout#1477](https://github.com/apache/mahout/issues/1477) are bimodal
(~5.5k vs ~11k on the same build). With `--repeats 3`, capture can land on the
fast mode and compare on the slow one, which reports a false REGRESSION at any
tolerance. Storing all repeats in the JSON and printing the spread on compare
would make that case recognisable. Not asking for a change to the default here,
just that the limitation is written in the module docstring.
##########
testing/qdp/test_parity.py:
##########
@@ -0,0 +1,202 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Cross-backend parity grid.
+
+``qumat_qdp.torch_ref`` is the specification for every encoding. This module
+runs the native Rust+CUDA engine over the full grid of
+
+ encoding x engine precision x input shape x input location
+
+and checks each cell against the reference. A new encoder is complete when it
+has a reference function and every cell here passes. A refactor of the engine
+or bindings is safe when this file is green before and after.
+"""
+
+from __future__ import annotations
+
+import math
+
+import numpy as np
+import pytest
+import torch
+
+from .qdp_test_utils import requires_qdp
+
+torch_ref = pytest.importorskip("qumat_qdp.torch_ref")
+
+NUM_QUBITS = 4
+BATCH = 5
+
+ENCODINGS = ("amplitude", "angle", "basis", "iqp", "iqp-z", "phase")
+PRECISIONS = ("float64", "float32")
+SHAPES = ("single", "batch")
+LOCATIONS = ("numpy", "torch_cpu", "cuda_f64", "cuda_f32")
+
+# Encodings with a device-side float32 kernel today. Others reject f32 CUDA
+# input, which is covered by ``test_f32_cuda_rejected_without_kernel``.
+F32_DEVICE_ENCODINGS = ("amplitude", "angle", "basis")
+
+# Encodings the current engine cannot read from a CUDA tensor at all. Cells
+# for these are expected failures until the single generic encode path lands;
+# once a listed encoding succeeds the cell fails with a message asking for the
+# entry to be removed, so the list cannot go stale.
+NO_DEVICE_PATH_ENCODINGS = ("phase",)
+
+# A cell's tolerance follows the narrowest precision on its path: the engine's
+# output precision, or float32 when the input arrives as a float32 CUDA tensor
+# and is encoded by the float32 kernel before any widening.
+TOL: dict[str, dict[str, float]] = {
+ "float64": {"atol": 1e-10, "rtol": 1e-9},
+ "float32": {"atol": 2e-5, "rtol": 2e-5},
+}
+
+
+def sample_dim(encoding: str, n: int) -> int:
+ if encoding == "amplitude":
+ return 1 << n
+ if encoding == "basis":
+ return 1
+ if encoding == "iqp":
+ return n + n * (n - 1) // 2
+ return n # angle, phase, iqp-z
+
+
+def make_samples(encoding: str, n: int, batch: int) -> np.ndarray:
+ """Deterministic float64 inputs of shape (batch, sample_dim)."""
+ rng = np.random.default_rng(1234 + len(encoding))
+ dim = sample_dim(encoding, n)
+ if encoding == "amplitude":
+ return rng.uniform(0.1, 1.0, size=(batch, dim))
+ if encoding == "basis":
+ return rng.integers(0, 1 << n, size=(batch, dim)).astype(np.float64)
+ return rng.uniform(0.0, 2.0 * math.pi, size=(batch, dim))
+
+
+def reference(encoding: str, data_2d: np.ndarray, n: int) -> torch.Tensor:
+ x = torch.from_numpy(data_2d).to(torch.float64)
+ return torch_ref.encode(x, n, encoding)
+
+
+def to_input(encoding: str, data_2d: np.ndarray, shape: str, location: str):
+ """Materialise the reference data as the requested input object."""
+ arr = data_2d if shape == "batch" else data_2d[0]
+ if location == "numpy":
+ return np.ascontiguousarray(arr)
+ t = torch.from_numpy(np.ascontiguousarray(arr))
+ if location == "torch_cpu":
+ return t
+ if location == "cuda_f64":
+ # Basis indices on the device are int64 by API contract; every other
+ # encoding takes float64.
+ return (t.to(torch.int64) if encoding == "basis" else t).cuda()
+ if location == "cuda_f32":
+ return t.to(torch.float32).cuda()
+ raise AssertionError(location)
+
+
+def engine_output(qt) -> torch.Tensor:
+ out = torch.from_dlpack(qt)
+ assert out.is_cuda
+ return out
+
+
+def cell_id(p):
Review Comment:
`cell_id` is identical to `str`; `ids=str` reads the same and removes a
helper.
##########
testing/qdp/test_parity.py:
##########
@@ -0,0 +1,202 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Cross-backend parity grid.
+
+``qumat_qdp.torch_ref`` is the specification for every encoding. This module
+runs the native Rust+CUDA engine over the full grid of
+
+ encoding x engine precision x input shape x input location
+
+and checks each cell against the reference. A new encoder is complete when it
+has a reference function and every cell here passes. A refactor of the engine
+or bindings is safe when this file is green before and after.
+"""
+
+from __future__ import annotations
+
+import math
+
+import numpy as np
+import pytest
+import torch
+
+from .qdp_test_utils import requires_qdp
+
+torch_ref = pytest.importorskip("qumat_qdp.torch_ref")
+
+NUM_QUBITS = 4
+BATCH = 5
+
+ENCODINGS = ("amplitude", "angle", "basis", "iqp", "iqp-z", "phase")
+PRECISIONS = ("float64", "float32")
+SHAPES = ("single", "batch")
+LOCATIONS = ("numpy", "torch_cpu", "cuda_f64", "cuda_f32")
+
+# Encodings with a device-side float32 kernel today. Others reject f32 CUDA
+# input, which is covered by ``test_f32_cuda_rejected_without_kernel``.
+F32_DEVICE_ENCODINGS = ("amplitude", "angle", "basis")
+
+# Encodings the current engine cannot read from a CUDA tensor at all. Cells
+# for these are expected failures until the single generic encode path lands;
+# once a listed encoding succeeds the cell fails with a message asking for the
+# entry to be removed, so the list cannot go stale.
+NO_DEVICE_PATH_ENCODINGS = ("phase",)
+
+# A cell's tolerance follows the narrowest precision on its path: the engine's
+# output precision, or float32 when the input arrives as a float32 CUDA tensor
+# and is encoded by the float32 kernel before any widening.
+TOL: dict[str, dict[str, float]] = {
+ "float64": {"atol": 1e-10, "rtol": 1e-9},
+ "float32": {"atol": 2e-5, "rtol": 2e-5},
+}
+
+
+def sample_dim(encoding: str, n: int) -> int:
+ if encoding == "amplitude":
+ return 1 << n
+ if encoding == "basis":
+ return 1
+ if encoding == "iqp":
+ return n + n * (n - 1) // 2
+ return n # angle, phase, iqp-z
+
+
+def make_samples(encoding: str, n: int, batch: int) -> np.ndarray:
+ """Deterministic float64 inputs of shape (batch, sample_dim)."""
+ rng = np.random.default_rng(1234 + len(encoding))
+ dim = sample_dim(encoding, n)
+ if encoding == "amplitude":
+ return rng.uniform(0.1, 1.0, size=(batch, dim))
+ if encoding == "basis":
+ return rng.integers(0, 1 << n, size=(batch, dim)).astype(np.float64)
+ return rng.uniform(0.0, 2.0 * math.pi, size=(batch, dim))
+
+
+def reference(encoding: str, data_2d: np.ndarray, n: int) -> torch.Tensor:
+ x = torch.from_numpy(data_2d).to(torch.float64)
+ return torch_ref.encode(x, n, encoding)
+
+
+def to_input(encoding: str, data_2d: np.ndarray, shape: str, location: str):
+ """Materialise the reference data as the requested input object."""
+ arr = data_2d if shape == "batch" else data_2d[0]
+ if location == "numpy":
+ return np.ascontiguousarray(arr)
+ t = torch.from_numpy(np.ascontiguousarray(arr))
+ if location == "torch_cpu":
+ return t
+ if location == "cuda_f64":
+ # Basis indices on the device are int64 by API contract; every other
+ # encoding takes float64.
+ return (t.to(torch.int64) if encoding == "basis" else t).cuda()
+ if location == "cuda_f32":
+ return t.to(torch.float32).cuda()
+ raise AssertionError(location)
+
+
+def engine_output(qt) -> torch.Tensor:
+ out = torch.from_dlpack(qt)
+ assert out.is_cuda
+ return out
+
+
+def cell_id(p):
+ return str(p)
+
+
+@requires_qdp
[email protected]
[email protected]("location", LOCATIONS, ids=cell_id)
[email protected]("shape", SHAPES, ids=cell_id)
[email protected]("precision", PRECISIONS, ids=cell_id)
[email protected]("encoding", ENCODINGS, ids=cell_id)
+def test_parity_grid(encoding, precision, shape, location):
+ from _qdp import QdpEngine
+
+ if location == "cuda_f32" and encoding not in F32_DEVICE_ENCODINGS:
+ pytest.skip(f"{encoding} has no float32 device kernel")
+
+ engine = QdpEngine(0, precision)
+ data = make_samples(encoding, NUM_QUBITS, BATCH)
+ # The engine always returns a (samples, 2**n) tensor, so a single sample
+ # is compared against the first row of the batch reference.
+ expected = reference(encoding, data, NUM_QUBITS)
+ if shape == "single":
+ expected = expected[:1]
+
+ inp = to_input(encoding, data, shape, location)
+ if location == "cuda_f64" and encoding in NO_DEVICE_PATH_ENCODINGS:
+ try:
+ engine.encode(inp, NUM_QUBITS, encoding)
+ except RuntimeError:
+ pytest.xfail(f"{encoding} has no device-pointer path yet")
+ pytest.fail(
+ f"{encoding} now encodes from a CUDA tensor; "
+ "remove it from NO_DEVICE_PATH_ENCODINGS"
+ )
+ out = engine_output(engine.encode(inp, NUM_QUBITS, encoding))
+
+ expected_dtype = torch.complex128 if precision == "float64" else
torch.complex64
+ assert out.dtype == expected_dtype
+ assert tuple(out.shape) == tuple(expected.shape)
+
+ # Float32 inputs on the device lose input precision before the kernel runs;
+ # compare against the reference evaluated on the same rounded inputs.
+ if location == "cuda_f32":
+ rounded = data.astype(np.float32).astype(np.float64)
+ expected = reference(encoding, rounded, NUM_QUBITS)
+ if shape == "single":
+ expected = expected[:1]
+
+ tol = TOL["float32"] if location == "cuda_f32" else TOL[precision]
+ torch.testing.assert_close(
+ out.cpu().to(torch.complex128),
+ expected,
+ atol=tol["atol"],
+ rtol=tol["rtol"],
+ )
+
+
+@requires_qdp
[email protected]
[email protected](
+ "encoding", [e for e in ENCODINGS if e not in F32_DEVICE_ENCODINGS]
+)
+def test_f32_cuda_rejected_without_kernel(encoding):
Review Comment:
For `phase` this passes because the binding rejects every CUDA tensor, not
because of a missing f32 kernel. The name is slightly misleading for that one
row; a comment or a separate `pytest.param` id would do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
