kotselidis opened a new pull request, #19251:
URL: https://github.com/apache/iotdb/pull/19251

   ## Description
   
   ### The problem
   
   `convert_tensor_to_tsblock` in `iotdb/ainode/core/util/serde.py` builds the 
TsBlock that AINode returns to the DataNode by appending to a `bytes` object 
one value at a time (`binary += value.tobytes()`). Each append copies the whole 
object, so the conversion is quadratic in the number of output values.
   
   It matters whenever a model returns a value per input point, for example a 
custom model that scores every point of a series through `CALL INFERENCE`. 
Measured in AINode's own environment (2.0.11, Python 3.11) for a `1 x n` 
float64 tensor:
   
   | n | Before |
   |--:|--:|
   | 10,000 | 0.084 s |
   | 20,000 | 0.32 s |
   | 40,000 | 1.28 s |
   | 80,000 | 17.9 s |
   | 160,000 | 45.0 s |
   
   End to end, a `CALL INFERENCE` that returns 100,000 per-point scores from a 
custom PyTorch model on a GPU took 10.1 s, of which 9.7 s was this conversion 
and 0.33 s the model. Results of a few hundred thousand points take minutes and 
run into `dn_connection_timeout_ms`, and the DataNode then retries the call.
   
   ### The change
   
   The function now takes the tensor's columns as a numpy array and builds the 
same layout from a list of parts joined once:
   
   - the header (column count, types, position count, encodings) as before;
   - the time column as one `bytes(8 * position_count)` block, which is what 
the previous loop wrote (two big-endian int32 zeros per row);
   - each value column as one big-endian `ndarray.tobytes()`.
   
   The output is byte-for-byte the same as before. The new 
`tests/test_serde.py` keeps the previous implementation as a reference and 
compares the two on scalars, 1-D and 2-D tensors, float32, float64, int32, 
int64 and bool, NaN, infinities and -0.0, a non-contiguous tensor, and a 2 x 
20,000 float64 tensor.
   
   After the change, the conversion takes 2 ms for 100,000 values and 5.5 ms 
for 1,000,000. The `CALL INFERENCE` above takes 0.34 s instead of 10.1 s.
   
   `struct` is no longer used and its import is removed; `pandas` is still used 
by the type helpers.
   
   ### Verification
   
   ```bash
   cd iotdb-core/ainode
   python -m unittest discover -s tests -p 'test_serde.py'
   black --check iotdb/ainode/core/util/serde.py tests/test_serde.py
   isort --check-only --profile black iotdb/ainode/core/util/serde.py 
tests/test_serde.py
   ```
   
   Run with AINode 2.0.11's environment (PyTorch 2.9.1), with Black 25.1.0 and 
isort 6.0.1. I have not run it on the PyTorch 2.14 environment that master's 
`pyproject.toml` now pins, nor run the AINode cluster integration tests.
   
   <hr>
   
   This PR has:
   - [x] been self-reviewed.
   - [x] added unit tests or modified existing tests to cover new code paths, 
ensuring the threshold for code coverage.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to