kotselidis opened a new pull request, #19272:
URL: https://github.com/apache/iotdb/pull/19272

   ## Description
   
   This PR adds [Timestral](https://github.com/mistralai/timestral) (Mistral 
AI) to AINode's built-in forecasting models, next to `timer_xl`, `sundial`, 
`chronos2`, `moirai2`, `toto` and `moment`. Timestral is a patch-based 
transformer for multivariate probabilistic forecasting. It forecasts 21 
quantiles (0.01 to 0.99) from a context of up to 32,768 steps, and it can use 
past-only and future-known covariates. The model is registered as `timestral`, 
with the weights of 
[`mistralai/Timestral-209M`](https://huggingface.co/mistralai/Timestral-209M). 
The 209M model is the largest of the four published sizes (5M, 28M, 123M, 209M) 
and the one Timestral's own documentation uses.
   
   Unlike the other built-in models, AINode does not download Timestral's 
weights at startup. They are downloaded when a user loads the model with `LOAD 
MODEL`. The reason is the weights' licence, below.
   
   ### Weights licence
   
   Timestral's code is under the Apache License 2.0. Its weights are not. Every 
published size is under a "Modified MIT" licence ([licence 
text](https://huggingface.co/mistralai/Timestral-209M/blob/main/LICENSE)), 
whose second condition reads:
   
   > You are not authorized to exercise any rights under this license if the 
global consolidated monthly revenue of your company (or that of your employer) 
exceeds $20 million (or its equivalent in another currency) for the preceding 
month.
   
   The weights are not part of the AINode distribution in either case. However, 
AINode downloads the weights of every built-in model when it starts. As an 
automatic built-in, Timestral would put weights that some users are not 
licensed to use on every AINode installation, without anyone asking for them. 
With this PR, the weights reach a machine only when someone runs `LOAD MODEL 
timestral`, which is a deliberate act by that user. The same question applies 
to `moirai2`, whose weights (`Salesforce/moirai-2.0-R-small`) are under CC 
BY-NC 4.0 and are downloaded at startup today. The new `download_on_start` flag 
could be applied to it as well. That is left to the maintainers, and this PR 
does not change `moirai2`.
   
   The maintainers may prefer a different policy, such as not shipping models 
with restricted weights at all. The Timestral integration does not depend on 
the on-request download, so it can follow whichever policy is chosen.
   
   ### Usage
   
   ```sql
   SHOW MODELS
   -- timestral | timestral | builtin | inactive   (weights not downloaded)
   
   SELECT * FROM FORECAST(model_id => 'timestral', targets => (SELECT time, s 
FROM db.t) ORDER BY time, output_length => 24)
   -- 1504: Model [timestral] is not downloaded at start. Use 'LOAD MODEL 
timestral TO DEVICES ...' to download and load it; 'SHOW MODELS' shows it as 
active once it is downloaded.
   
   LOAD MODEL timestral TO DEVICES 'cpu'
   -- downloads config.json and model.safetensors from 
mistralai/Timestral-209M, then loads the model
   
   SHOW MODELS
   -- timestral | timestral | builtin | active
   
   SELECT * FROM FORECAST(model_id => 'timestral', targets => (SELECT time, s 
FROM db.t) ORDER BY time, output_length => 24)
   -- 24 forecast rows
   ```
   
   Once downloaded, the weights stay in AINode's model directory, and the model 
is active from the next start on.
   
   Example: the input is 1024 points of `50 + 10·sin(2πt/24) + 0.01·t`. The 
request is a `TForecastReq` with `outputLength = 24`, sent to AINode's 
inference manager on CPU with the Timestral-209M checkpoint. The response is 24 
values (the median forecast). The first six:
   
   | step | forecast | true value |
   |---|---|---|
   | 1 | 51.507 | 51.580 |
   | 2 | 50.514 | 50.591 |
   | 3 | 50.168 | 50.260 |
   | 4 | 50.564 | 50.611 |
   | 5 | 51.589 | 51.620 |
   | 6 | 53.205 | 53.219 |
   
   The mean absolute error is 0.031 over 24 steps and 0.049 over 96 steps.
   
   ### On-request download of built-in models
   
   - `ModelInfo` gets a `download_on_start` flag, which is `True` by default. 
Every existing built-in model keeps it, so their behaviour does not change.
   - `ModelStorage` registers a built-in model with `download_on_start=False` 
as `inactive` at startup, unless its `config.json` and `model.safetensors` are 
already in its directory. `ModelStorage.download_builtin_model` downloads the 
two files and marks the model `active`, or `inactive` again if the download 
fails. The download code is the existing one, moved into a method.
   - `LOAD MODEL` runs the download in the pool controller's background load 
task, before the inference pools start. The `LOAD MODEL` statement returns 
immediately, as it does today. `SHOW MODELS` shows `activating` during the 
download, and `SHOW LOADED MODELS` shows the model once it is loaded.
   - `FORECAST` and `CALL INFERENCE` on such a model before the download return 
error 1504 with the message shown above. Without this check, they would fail 
while loading the missing weights.
   
   ### Model code
   
   `iotdb/ainode/core/model/timestral/` contains the inference code from 
mistralai/timestral 1.0.0 (Apache License 2.0), in the same way as `chronos2/` 
contains the code from chronos-forecasting. The model, forecaster, layers, 
masks, scaler, covariate encoding and configuration are unchanged except for 
these points:
   
   - Imports use the `iotdb.ainode.core.model.timestral` package, and files 
carry the ASF header followed by Mistral AI's copyright notice.
   - The Hugging Face download helper (`loading.py`) and 
`TimestralForecaster.from_pretrained` are left out. AINode downloads and loads 
the checkpoint itself.
   
   ### AINode integration
   
   - `modeling_timestral.TimestralForPrediction` is the model, loaded through 
`PyTorchModelHubMixin` (`hub_mixin_cls` in `ModelInfo`), a loading path AINode 
already supports. `config.json` is read into Timestral's own `TimestralConfig`, 
which rejects unknown keys. The weights load with `strict=True`. The 
rotary-embedding frequencies are recomputed whenever the model is moved to a 
device, as Timestral's own loader does after moving the model.
   - `pipeline_timestral.TimestralPipeline` runs Timestral's 
`TimestralForecaster` with its default settings:
     - Each input is one series, and its targets are forecast jointly.
     - A covariate that appears only in `past_covariates` becomes a past 
covariate. A covariate that also appears in `future_covariates` becomes a known 
covariate, built from its past and future values.
     - Inputs of the same shape without covariates run in one batch.
     - Forecasts are averaged with the reflected forecast of the negated 
series, and are clipped at 0 for series whose context is non-negative.
     - Horizons longer than one decode block (1024 steps) are forecast block by 
block, so any `output_length` that AINode accepts (up to 2880) works.
     - The pipeline returns the 0.5 quantile.
   
   ### Dependencies
   
   There are no new dependencies. Timestral needs only PyTorch, NumPy, 
safetensors and huggingface_hub, which AINode already pins. `pyproject.toml`, 
`ainode.spec` and `ainode.xml` are unchanged. `ainode.spec` already collects 
every module under `iotdb.ainode.core`.
   
   ### Licences
   
   The code is under the Apache License 2.0 
([licence](https://github.com/mistralai/timestral/blob/main/LICENSE)). This PR 
adds it to `LICENSE`, `NOTICE` and AINode's `NOTICE-binary`. See the note above 
about the licence of the weights.
   
   ### Tests
   
   - `tests/test_timestral.py` (unittest, CPU only, no downloads) uses a 
two-layer Timestral with random weights. The tests:
     - save it as a checkpoint and check that it reloads with identical weights;
     - check that a checkpoint with a missing weight, or a config with an 
unknown key, is rejected;
     - check that the rotary-embedding frequencies are rebuilt when the model 
is cast;
     - load it through AINode's built-in loader and the `TimestralPipeline`, 
and check the output shapes for 1-D and 2-D targets over more than one decode 
block, and that the result is the forecaster's median;
     - check how covariates are assembled and that covariates change the 
forecast.
   - `tests/test_builtin_model_download.py` (unittest, no network: 
`hf_hub_download` is mocked) checks that:
     - Timestral is `inactive` and not downloaded at startup, while the other 
built-ins are;
     - `download_builtin_model` fetches it and marks it `active`, and a second 
call downloads nothing;
     - it is `active` at the next startup once downloaded;
     - a failed download leaves it `inactive`;
     - forecasts before the download are rejected, and other models are not 
affected.
   - Integration tests: `AINodeTestUtils.ON_REQUEST_BUILTIN_MODEL_MAP` lists 
`timestral`. The `SHOW MODELS` test accepts it in either state, and the new 
`onRequestBuiltInModelLoadInferenceAndForecastTest` loads it on the CPU, which 
downloads it, then runs `CALL INFERENCE` and `FORECAST`.
   - A run of AINode's RPC handler on CPU confirms the whole flow. `SHOW 
MODELS` first reports `inactive`, and `FORECAST` returns 1504. `LOAD MODEL 
timestral TO DEVICES 'cpu'` downloads the files and starts one request pool. 
`FORECAST` then returns the 24 values above through that pool.
   - With the published checkpoints on CPU, the forecasts are bit-identical to 
the unmodified Timestral 1.0.0 code for a multivariate series, a 1100-step 
forecast spanning two decode blocks, and a series with past and known 
covariates.
   
   <hr>
   
   This PR has:
   - [x] been self-reviewed.
   - [x] added or updated version, __license__, or notice information
   - [x] added comments explaining the "why" and the intent of the code 
wherever would not be obvious
     for an unfamiliar reader.
   - [x] added unit tests or modified existing tests to cover new code paths, 
ensuring the threshold
     for code coverage.
   - [x] added integration tests.
   - [x] added documentation for new or modified features or behaviors.
   
   <hr>
   
   ##### Key changed/added classes (or packages if there are too many classes) 
in this PR
   
   - `iotdb/ainode/core/model/timestral/` (new): the Timestral model and 
forecaster, `TimestralForPrediction`, `TimestralPipeline`
   - `iotdb/ainode/core/model/model_info.py`: `ModelInfo.download_on_start`; 
`timestral` entry in `BUILTIN_HF_TRANSFORMERS_MODEL_MAP`
   - `iotdb/ainode/core/model/model_storage.py`: `download_builtin_model`; 
built-ins with `download_on_start=False` are not downloaded at start
   - `iotdb/ainode/core/inference/pool_controller.py`: `LOAD MODEL` downloads 
such a model before loading it
   - `iotdb/ainode/core/rpc/handler.py`: forecasts on a model that is not 
downloaded yet are rejected
   - `iotdb-core/ainode/tests/test_timestral.py`, 
`tests/test_builtin_model_download.py` (new)
   - `integration-test/.../ainode/utils/AINodeTestUtils.java`, 
`.../it/AINodeSharedClusterIT.java`: `ON_REQUEST_BUILTIN_MODEL_MAP` and its 
tests
   - `LICENSE`, `NOTICE`, 
`iotdb-core/ainode/src/assembly/resources/NOTICE-binary`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to