[
https://issues.apache.org/jira/browse/IOTDB-6081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Yurong Su reassigned IOTDB-6081:
--------------------------------------
Assignee: Steve Yurong Su
> Pipe: use HybridExtractor instead of LogExtractor when realtime mode is set
> to log to avoid OOM under heavy insertion load
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: IOTDB-6081
> URL: https://issues.apache.org/jira/browse/IOTDB-6081
> Project: Apache IoTDB
> Issue Type: Bug
> Reporter: Steve Yurong Su
> Assignee: Steve Yurong Su
> Priority: Major
> Attachments: image-2023-07-24-11-54-50-433.png
>
>
> Scenarios:
> 1. source iotdb: 3C3D 2 replica， target iotdb: 1C1D
> 2. create and start 10 realtime pipes with log mode enabled
> {code:java}
> create pipe pipe_db_1 with extractor ('extractor'='iotdb-extractor',
> 'extractor.pattern'='root.test.g_0','extractor.realtime.enable'='true','extractor.realtime.mode'='log')
> with connector ('connector'='iotdb-thrift-connector',
> 'connector.ip'='172.20.70.15', 'connector.port'='6667');
> {code}
> 4. start benchmark: device: 100 sensor: 10 database: 1 aligned timeseries
> !image-2023-07-24-11-54-50-433.png|width=863,height=415!
>
> Fixes introcuded:
> * When user specifies 'extractor.realtime.mode'='log', we use hybrid
> extractors which can automatically degrade to file mode when load is too
> heavy.
> * Add a parameter: 'extractor.realtime.mode'='forced-log', the pipe will
> always use log mode and will never degrade to file mode.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
