Eliaaazzz opened a new pull request, #17386:
URL: https://github.com/apache/iotdb/pull/17386
## Description
### Integrate MOMENT Time Series Foundation Model
Integrate [MOMENT](https://arxiv.org/abs/2402.03885) (A Family of Open
Time-series Foundation Models, ICML 2024, Auton Lab/CMU) into IoTDB AINode as a
built-in forecasting model.
MOMENT uses a T5 encoder-only backbone with patch-based input embedding
and RevIN (Reversible Instance Normalization). It supports zero-shot
forecasting with a fixed input length of 512 timesteps
and channel-independent multivariate processing.
Architecture:
Input [batch, n_channels, 512]
→ RevIN normalization
→ Patching (patch_len=8, stride=8 → 64 patches)
→ Patch embedding (linear → d_model)
→ T5 Encoder (self-attention)
→ ForecastingHead (linear → forecast_horizon)
→ RevIN denormalization
→ Output [batch, n_channels, forecast_horizon]
The default variant is MOMENT-1-small (~152 MB, d_model=512, 6 layers).
Weights are downloaded from HuggingFace Hub on first use, consistent with
existing built-in models (Sundial, Timer-XL,
Chronos2, etc.).
### Design Decisions
- **Vendor the model code** rather than depending on the external
`momentfm` pip package, following the same pattern as Chronos2 (#16903) and
Toto (#17322). This avoids an additional runtime
dependency and gives full control over the loading process.
- **Wrap as `PreTrainedModel`** with custom `from_pretrained()` to handle
upstream MOMENT's flat state-dict key mapping (prefixing with `moment.*`) and
T5 config extraction from `config.json`.
- **Use MOMENT-1-small** as the default built-in variant to minimize
download size and memory usage while still providing useful forecasting
capability.
- **Iterative long-horizon forecasting** in the pipeline: for
`output_length > model_forecast_horizon`, predictions are appended to the
context window and fed back as input, similar to
autoregressive decoding.
- **Fixed 512-length input handling**: inputs shorter than 512 are
left-padded with zeros and masked; inputs longer than 512 are truncated to the
last 512 timesteps.
### Bug Fix
Fixed unquoted `$JAVA` variable in `confignode-env.sh` and
`datanode-env.sh` to handle paths with spaces (e.g. `C:\Program Files\...`).
<hr>
This PR has:
- [x] been self-reviewed.
- [x] added or updated version, __license__, or notice information
- [x] added comments explaining the "why" and the intent of the code
wherever would not be obvious for an unfamiliar reader.
- [x] added integration tests.
- [x] been tested in a test IoTDB cluster.
<hr>
##### Key changed/added classes (or packages if there are too many
classes) in this PR
- `iotdb-core/ainode/iotdb/ainode/core/model/moment/` — New MOMENT model
package
- `configuration_moment.py` — `MomentConfig(PretrainedConfig)`
- `modeling_moment.py` — `MomentForPrediction(PreTrainedModel)` with
RevIN, Patching, T5 Encoder, ForecastingHead
- `pipeline_moment.py` — `MomentPipeline(ForecastPipeline)` with
padding/truncation and iterative forecasting
- `iotdb-core/ainode/iotdb/ainode/core/model/model_info.py` — Added
`moment` to `BUILTIN_HF_TRANSFORMERS_MODEL_MAP`
- `iotdb-core/ainode/iotdb/ainode/core/constant.py` — Added memory
estimate for MOMENT
- `integration-test/.../AINodeTestUtils.java` — Added `moment` to
`BUILTIN_LTSM_MAP`
- `LICENSE` — Added MIT attribution for MOMENT source code
- `scripts/conf/confignode-env.sh`, `scripts/conf/datanode-env.sh` — Fixed
unquoted `$JAVA` variable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
