[jira] [Created] (IOTDB-3026) [load] cannot load tsfile correctly after restart iotdb
yanze chen created IOTDB-3026: - Summary: [load] cannot load tsfile correctly after restart iotdb Key: IOTDB-3026 URL: https://issues.apache.org/jira/browse/IOTDB-3026 Project: Apache IoTDB Issue Type: Bug Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3047) [sync] Path does not exist when delete storage group
yanze chen created IOTDB-3047:
-
Summary: [sync] Path does not exist when delete storage group
Key: IOTDB-3047
URL: https://issues.apache.org/jira/browse/IOTDB-3047
Project: Apache IoTDB
Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen
34062 [pool-19-IoTDB-Sync-Collector-1] INFO o.a.i.d.s.r.collector.Collector -
Start load pipeData with serialize number 25 and type
SCHEMA,value=SchemaPipeData\{serialNumber=25,
plan=DeleteTimeSeriesPlan{deletePathList=[root.sg2.d1, root.sg2.d1.**,
root.sg2.d2, root.sg2.d2.**], results={}, partitionFilter=null}}
34062 [pool-19-IoTDB-Sync-Collector-1] ERROR o.a.i.d.s.r.collector.Collector -
Cannot load pipeData with serialize number 25 and type SCHEMA, because Path
[root.sg2.d1] does not exist
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (IOTDB-3110) [sync] drop and create pipe throws NullPointerException
yanze chen created IOTDB-3110: - Summary: [sync] drop and create pipe throws NullPointerException Key: IOTDB-3110 URL: https://issues.apache.org/jira/browse/IOTDB-3110 Project: Apache IoTDB Issue Type: Bug Reporter: yanze chen Assignee: yanze chen Attachments: WechatIMG1552.png 步骤 1.发送端create pipesink i 2.发送端create pipe p to i 3.发送端drop pipe i 4.发送端create pipe p to i -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3144) [sync] Set the default pipe status of receiver after reboot to STOP
yanze chen created IOTDB-3144: - Summary: [sync] Set the default pipe status of receiver after reboot to STOP Key: IOTDB-3144 URL: https://issues.apache.org/jira/browse/IOTDB-3144 Project: Apache IoTDB Issue Type: Bug Reporter: yanze chen Assignee: yanze chen Attachments: 状态.jpeg Find the detail in attachment -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3130) [0.14][new_sync] Be interrupted when waiting for pipe data, because null
[ https://issues.apache.org/jira/browse/IOTDB-3130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanze chen reassigned IOTDB-3130: - Assignee: yanze chen (was: yusicheng) > [0.14][new_sync] Be interrupted when waiting for pipe data, because null > - > > Key: IOTDB-3130 > URL: https://issues.apache.org/jira/browse/IOTDB-3130 > Project: Apache IoTDB > Issue Type: Bug >Reporter: ZhangZhengming >Assignee: yanze chen >Priority: Minor > Attachments: image-2022-05-09-18-39-46-140.png, > image-2022-05-09-19-12-45-500.png > > > 1. In iotdb sender， start pipe > 2. Iotdb sender exit with error > kill -9 [iotdb pid] > 3. In receiver, report a null > !image-2022-05-09-19-12-45-500.png|width=1223,height=72! > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (IOTDB-3130) [0.14][new_sync] Be interrupted when waiting for pipe data, because null
[ https://issues.apache.org/jira/browse/IOTDB-3130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17535955#comment-17535955 ] yanze chen commented on IOTDB-3130: --- https://github.com/apache/iotdb/pull/5859 > [0.14][new_sync] Be interrupted when waiting for pipe data, because null > - > > Key: IOTDB-3130 > URL: https://issues.apache.org/jira/browse/IOTDB-3130 > Project: Apache IoTDB > Issue Type: Bug >Reporter: ZhangZhengming >Assignee: yanze chen >Priority: Minor > Attachments: image-2022-05-09-18-39-46-140.png, > image-2022-05-09-19-12-45-500.png > > > 1. In iotdb sender， start pipe > 2. Iotdb sender exit with error > kill -9 [iotdb pid] > 3. In receiver, report a null > !image-2022-05-09-19-12-45-500.png|width=1223,height=72! > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3190) [Design] sync adaptation to mpp framework
yanze chen created IOTDB-3190: - Summary: [Design] sync adaptation to mpp framework Key: IOTDB-3190 URL: https://issues.apache.org/jira/browse/IOTDB-3190 Project: Apache IoTDB Issue Type: Bug Reporter: yanze chen Assignee: yanze chen https://apache-iotdb.feishu.cn/docs/doccnJ6PdRBRDjlbgU8R3MoQdqf#Vn0OuH -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3191) implement sync receiver pipe manager in mpp framework
yanze chen created IOTDB-3191: - Summary: implement sync receiver pipe manager in mpp framework Key: IOTDB-3191 URL: https://issues.apache.org/jira/browse/IOTDB-3191 Project: Apache IoTDB Issue Type: Bug Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3192) implement sync receiver data collector in mpp framework
yanze chen created IOTDB-3192: - Summary: implement sync receiver data collector in mpp framework Key: IOTDB-3192 URL: https://issues.apache.org/jira/browse/IOTDB-3192 Project: Apache IoTDB Issue Type: Bug Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3133) [0.14][new_sync] When two iotdb die at the same time, the pipe not workde
[
https://issues.apache.org/jira/browse/IOTDB-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yanze chen reassigned IOTDB-3133:
-
Assignee: yanze chen (was: yusicheng)
> [0.14][new_sync] When two iotdb die at the same time, the pipe not workde
> -
>
> Key: IOTDB-3133
> URL: https://issues.apache.org/jira/browse/IOTDB-3133
> Project: Apache IoTDB
> Issue Type: Bug
>Reporter: ZhangZhengming
>Assignee: yanze chen
>Priority: Blocker
> Attachments: image-2022-05-09-19-58-18-007.png,
> image-2022-05-09-19-58-52-579.png, image-2022-05-09-20-22-17-420.png
>
>
> # start pipe
> # kill all iotdb
> # start sender
> # {color:#ff}show pipe report an error, "can not connect"{color}
> ** !image-2022-05-09-19-58-52-579.png|width=1137,height=111!
> ** And, this status is {color:#ff}"RUNNING"{color}
> # start receiver
> ** {color:#ff}He can't reconnect automatically{color}
> # Then, i execute "stop pipe,start pipe "
> # The pipe is worked。but in fact, he is not working， I found this error in
> receiver
> ##
> {color:#ff}!image-2022-05-09-20-22-17-420.png|width=858,height=56!{color}
> # I've been waiting for a long time, but it still doesn't work
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (IOTDB-3477) Send two phase message to ConfigNode when starting DataNode
yanze chen created IOTDB-3477: - Summary: Send two phase message to ConfigNode when starting DataNode Key: IOTDB-3477 URL: https://issues.apache.org/jira/browse/IOTDB-3477 Project: Apache IoTDB Issue Type: Bug Reporter: yanze chen Assignee: yanze chen https://apache-iotdb.feishu.cn/docs/doccnMgSBOgHqeT9gjs4RUrqTob -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3515) load empty TsFile error message
yanze chen created IOTDB-3515: - Summary: load empty TsFile error message Key: IOTDB-3515 URL: https://issues.apache.org/jira/browse/IOTDB-3515 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen Fix For: master branch, 0.13.1, 0.12.5 IoTDB> load '/Users/chenyanze/Downloads/1654557520768-5-0-0/1654557520768-5-0-0.tsfile' Msg: 411: Cannot load file /Users/chenyanze/Downloads/1654557520768-5-0-0/1654557520768-5-0-0.tsfile because Can not get the corresponding storage group. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (IOTDB-3398) Device not in TsFile
[ https://issues.apache.org/jira/browse/IOTDB-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556936#comment-17556936 ] yanze chen commented on IOTDB-3398: --- https://apache-iotdb.feishu.cn/docx/doxcnRjJpShOWSPCZgpt4pb42Ee > Device not in TsFile > > > Key: IOTDB-3398 > URL: https://issues.apache.org/jira/browse/IOTDB-3398 > Project: Apache IoTDB > Issue Type: Bug >Affects Versions: 0.12.5 >Reporter: Jialin Qiao >Assignee: yanze chen >Priority: Major > Attachments: iotdblost.zip > > > > When querying a TsFile, we suspect that the device should be in this file. > Howerver, it doesn't. > > 2022-06-05 14:02:19,048 [pool-37-IoTDB-Sub_RawQuery-5] WARN > o.a.i.t.r.TsFileSequenceReader:336 - Device root.sg.track19.d_53810829218 is > not in /mntb/iotdb/data/sequence/root.sg/1/0/1654406867872-6737-0-0.tsfile -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3602) Skip empty TsFile when loading directory
yanze chen created IOTDB-3602: - Summary: Skip empty TsFile when loading directory Key: IOTDB-3602 URL: https://issues.apache.org/jira/browse/IOTDB-3602 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen load整个文件夹时，有个空文件导致剩下的文件没load进来 优化一下这里，遇到报错的文件跳过就行，不影响剩余的文件 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (IOTDB-3247) [Persistent schema] [wal recovery] Aligned sensors, query lost data
Title: Message Title yanze chen commented on IOTDB-3247 Re: [Persistent schema] [wal recovery] Aligned sensors, query lost data 一个简单的复现方法： 1）IoTDB 默认配置启动 2）bm配置： SENSOR_NUMBER=10 DEVICE_NUMBER=1 3）执行以下语句 delete timeseries root.test.g_0.test.schema_t.d_0.s_0 delete timeseries root.test.g_0.test.schema_t.d_0.s_9 4）kill iotdb进程 kill -9 pid 5）重启IoTDB，报错 注：直接把附件中的data放到IOTDB目录下代替步骤1～4，启动即可 Add Comment This message was sent by Atlassian Jira (v8.20.10#820010-sha1:ace47f9)
[jira] [Created] (IOTDB-3730) ArrayIndexOutOfBounds when flushing a memtable
yanze chen created IOTDB-3730: - Summary: ArrayIndexOutOfBounds when flushing a memtable Key: IOTDB-3730 URL: https://issues.apache.org/jira/browse/IOTDB-3730 Project: Apache IoTDB Issue Type: Bug Affects Versions: 0.12.5 Reporter: yanze chen Assignee: yanze chen [https://github.com/apache/iotdb/issues/6551] [https://github.com/apache/iotdb/pull/6558] int转float精度丢失导致 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-3788) insertRecord support insert Binary as TEXT DataType
yanze chen created IOTDB-3788: - Summary: insertRecord support insert Binary as TEXT DataType Key: IOTDB-3788 URL: https://issues.apache.org/jira/browse/IOTDB-3788 Project: Apache IoTDB Issue Type: Sub-task Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-3789) Research and design documentation
yanze chen created IOTDB-3789: - Summary: Research and design documentation Key: IOTDB-3789 URL: https://issues.apache.org/jira/browse/IOTDB-3789 Project: Apache IoTDB Issue Type: Sub-task Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-3790) Implementing the new TsDataType BINARY
yanze chen created IOTDB-3790: - Summary: Implementing the new TsDataType BINARY Key: IOTDB-3790 URL: https://issues.apache.org/jira/browse/IOTDB-3790 Project: Apache IoTDB Issue Type: Sub-task Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-3803) failed to insert data of TEXT by session
[ https://issues.apache.org/jira/browse/IOTDB-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanze chen reassigned IOTDB-3803: - Assignee: yanze chen (was: Yongzao Dan) > failed to insert data of TEXT by session > > > Key: IOTDB-3803 > URL: https://issues.apache.org/jira/browse/IOTDB-3803 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Cluster >Affects Versions: 0.13.1-SNAPSHOT >Reporter: 刘俊男 >Assignee: yanze chen >Priority: Critical > Attachments: image-2022-07-12-11-59-15-955.png, > image-2022-07-12-11-59-22-491.png > > > !image-2022-07-12-11-59-22-491.png! > failed to use 'session.insertTablet(tablet)' to insert String type instead > of binary -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-3190) [Design] sync adaptation to mpp framework
[ https://issues.apache.org/jira/browse/IOTDB-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanze chen reassigned IOTDB-3190: - Assignee: yanze chen (was: yusicheng) > [Design] sync adaptation to mpp framework > - > > Key: IOTDB-3190 > URL: https://issues.apache.org/jira/browse/IOTDB-3190 > Project: Apache IoTDB > Issue Type: Task >Reporter: yanze chen >Assignee: yanze chen >Priority: Major > > https://apache-iotdb.feishu.cn/docs/doccnJ6PdRBRDjlbgU8R3MoQdqf#Vn0OuH -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-3805) Document and example of TEXT session interface
yanze chen created IOTDB-3805: - Summary: Document and example of TEXT session interface Key: IOTDB-3805 URL: https://issues.apache.org/jira/browse/IOTDB-3805 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen Fix For: master branch -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-3816) StorageEngine failed to recover: / by zero
[
https://issues.apache.org/jira/browse/IOTDB-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566313#comment-17566313
]
yanze chen commented on IOTDB-3816:
---
问题排查：https://apache-iotdb.feishu.cn/docx/doxcnRUNBYxmamBCr0Sy3SAiV0M
> StorageEngine failed to recover: / by zero
> --
>
> Key: IOTDB-3816
> URL: https://issues.apache.org/jira/browse/IOTDB-3816
> Project: Apache IoTDB
> Issue Type: Bug
>Affects Versions: master branch, 0.13.0, 0.12.5
>Reporter: Jialin Qiao
>Assignee: yanze chen
>Priority: Major
> Labels: pull-request-available
>
> When recover system, some memtable generated by wal may do not have valid
> series, cause / by zero in a logger.
> {code:java}
> 2022-07-13 14:47:14,488 [Thread-9] ERROR
> o.a.i.d.c.IoTDBDefaultThreadExceptionHandler:31 - Exception in thread
> Thread-9-47
> org.apache.iotdb.db.exception.runtime.StorageEngineFailureException:
> StorageEngine failed to recover.
> at
> org.apache.iotdb.db.engine.StorageEngine.lambda$recover$1(StorageEngine.java:245)
> at java.base/java.lang.Thread.run(Thread.java:830)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.ArithmeticException: / by zero
> at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at
> org.apache.iotdb.db.engine.StorageEngine.lambda$recover$1(StorageEngine.java:243)
> ... 1 common frames omitted
> Caused by: java.lang.ArithmeticException: / by zero
> at
> org.apache.iotdb.db.engine.flush.MemTableFlushTask.syncFlushMemTable(MemTableFlushTask.java:99)
> at
> org.apache.iotdb.db.writelog.recover.TsFileRecoverPerformer.redoLogs(TsFileRecoverPerformer.java:299)
> at
> org.apache.iotdb.db.writelog.recover.TsFileRecoverPerformer.recover(TsFileRecoverPerformer.java:143)
> at
> org.apache.iotdb.db.engine.storagegroup.VirtualStorageGroupProcessor.recoverTsFiles(VirtualStorageGroupProcessor.java:785)
> at
> org.apache.iotdb.db.engine.storagegroup.VirtualStorageGroupProcessor.recover(VirtualStorageGroupProcessor.java:526)
> at
> org.apache.iotdb.db.engine.storagegroup.VirtualStorageGroupProcessor.(VirtualStorageGroupProcessor.java:403)
> at
> org.apache.iotdb.db.engine.StorageEngine.buildNewStorageGroupProcessor(StorageEngine.java:596)
> at
> org.apache.iotdb.db.engine.storagegroup.virtualSg.StorageGroupManager.lambda$asyncRecover$0(StorageGroupManager.java:244)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> ... 1 common frames omitted {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IOTDB-3948) Sync receiver-side statement generation and standalone execution process
yanze chen created IOTDB-3948: - Summary: Sync receiver-side statement generation and standalone execution process Key: IOTDB-3948 URL: https://issues.apache.org/jira/browse/IOTDB-3948 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-3949) Sync data collection process in standalone version
yanze chen created IOTDB-3949: - Summary: Sync data collection process in standalone version Key: IOTDB-3949 URL: https://issues.apache.org/jira/browse/IOTDB-3949 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-3950) Transport layer optimization for sync modules
yanze chen created IOTDB-3950: - Summary: Transport layer optimization for sync modules Key: IOTDB-3950 URL: https://issues.apache.org/jira/browse/IOTDB-3950 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen https://apache-iotdb.feishu.cn/docx/doxcnVFJaPxG9bHQo2K5Y10sHxh -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-3989) Server thread does not release
[ https://issues.apache.org/jira/browse/IOTDB-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572998#comment-17572998 ] yanze chen commented on IOTDB-3989: --- It has been verified that the thread count of IOTDB does not remain at a high level after closing the SessionPress program. During the execution of the SessionPress program, IOTDB creates threads for each session. When the SessionPress program is stopped, the resources are immediately released and the threads transfer to the TIMED_WAITING state( as shown in the attachment `rpc_client_thread_state.png` ), and by default these idle threads are released after 60 seconds. `IoTDBConfig.java` has a `thriftServerAwaitTimeForStopService` parameter which will be used when the Thrift service is created and eventually as the keepAliveTime parameter in the ThreadPoolExecutor. The release time of idle threads can be controlled by adjusting the thriftServerAwaitTimeForStopService. For example, if you set thriftServerAwaitTimeForStopService to 5, then 5 seconds after the SessionPress application exits, the idle threads will be released and a drop in the number of threads can be observed. > Server thread does not release > -- > > Key: IOTDB-3989 > URL: https://issues.apache.org/jira/browse/IOTDB-3989 > Project: Apache IoTDB > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: 刘爽 >Assignee: yanze chen >Priority: Major > Attachments: SessionPress.java, image-2022-07-28-15-11-45-684.png, > image-2022-07-28-15-12-12-341.png, rpc_client_thread_state.png > > > (1) Run SessionPress > (2) Kill the SessionPress > The server thread does not release. > > The client stops abnormally，the Server thread does not release > !image-2022-07-28-15-12-12-341.png! > !image-2022-07-28-15-11-45-684.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-3920) [ 0.13.1 ] stop-server.sh , iotdb process does not exit
[
https://issues.apache.org/jira/browse/IOTDB-3920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17573000#comment-17573000
]
yanze chen commented on IOTDB-3920:
---
as same as https://issues.apache.org/jira/browse/IOTDB-3813
> [ 0.13.1 ] stop-server.sh , iotdb process does not exit
> ---
>
> Key: IOTDB-3920
> URL: https://issues.apache.org/jira/browse/IOTDB-3920
> Project: Apache IoTDB
> Issue Type: Bug
>Affects Versions: 0.13.1-SNAPSHOT
>Reporter: 刘珍
>Assignee: yanze chen
>Priority: Major
> Attachments: insert_is_overflow.config-2.properties, log_all.log
>
>
> rel/0.13 23856d20a22ae3c3260d7b318151307a3f277bd9
> {color:#DE350B}对齐序列+ template{color} ，stop-server.sh，iotdb进程不退出：
> 1. benchmark运行附件中的脚本：insert_is_overflow.config.properties
> 2. benchmark写入完成，调用脚本停止数据库：
> 先执行flush
> start-cli.sh -e "flush"
> 再调用stop-server.sh，sleep 10秒，如果iotdb进程还存在，就kill -9 ：
> stop-server.sh
> sleep 10
> iotdb_pid=`jps|grep IoTDB |awk '
> {print $1}
> '`
> if [ "${iotdb_pid}x" = "x" ];then
> echo "stop iotdb server successed."
> else
> echo "stop iotdb server failed."
> echo "stop iotdb server failed." >> ${iotdb_dir}/logs/stop_iotdb.log
> kill -9 ${iotdb_pid}
> fi
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IOTDB-3950) Transport layer optimization for sync modules
[ https://issues.apache.org/jira/browse/IOTDB-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576658#comment-17576658 ] yanze chen commented on IOTDB-3950: --- No more rpc in sync, use InternalService（see review comment of https://github.com/apache/iotdb/pull/6817） > Transport layer optimization for sync modules > - > > Key: IOTDB-3950 > URL: https://issues.apache.org/jira/browse/IOTDB-3950 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: yanze chen >Assignee: yanze chen >Priority: Major > > [https://apache-iotdb.feishu.cn/docx/doxcnVFJaPxG9bHQo2K5Y10sHxh] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-4022) Adjust the storage_group_report_threshold according to memory for writing
[ https://issues.apache.org/jira/browse/IOTDB-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17577697#comment-17577697 ] yanze chen commented on IOTDB-4022: --- h2. 测试环境 IOTDB：私有云 16核32G 1T的硬盘，默认配置，根据日志看到写入内存分配=2993055334 benchmark：周测配置aligned_seq_config.properties、unseq_config.properties h2. 测试结果 || ||aligned_seq_config.properties||unseq_config.properties|| |storage_group_report_threshold=16777216|2209.05 second|1598.57 second| |write_memory_vriation_report_proportion=0.001|2361.42 second|1632.44 second| |write_memory_vriation_report_proportion=0.01|2190.96 second| | > Adjust the storage_group_report_threshold according to memory for writing > - > > Key: IOTDB-4022 > URL: https://issues.apache.org/jira/browse/IOTDB-4022 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Jialin Qiao >Assignee: yanze chen >Priority: Major > Labels: pull-request-available > > Currently, we have a parameter called storage_group_report_threshold, which > means when a storage group report its memory usage, it should accumulate a > memory change of this number. > If this parameter is too small, may report frequently, which may decrease the > insertion performance. > If this parameter is too large, may report not in time, which may cause OOM. > > We need to change this parameter according to the memory allocated for > writing. > > (1) remove storage_group_report_threshold > (2) Add a new parameter write_memory_vriation_report_proportion > > To rel/0.13 and master -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4116) [sync] Merge SenderService and ReceiverService into SyncService
yanze chen created IOTDB-4116: - Summary: [sync] Merge SenderService and ReceiverService into SyncService Key: IOTDB-4116 URL: https://issues.apache.org/jira/browse/IOTDB-4116 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen Merge SenderService and ReceiverService into SyncService -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4135) [sync] Merge thrift-rpc into ClientRPC
yanze chen created IOTDB-4135: - Summary: [sync] Merge thrift-rpc into ClientRPC Key: IOTDB-4135 URL: https://issues.apache.org/jira/browse/IOTDB-4135 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen No more rpc in sync, use InternalService（see review comment of [https://github.com/apache/iotdb/pull/6817]） -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4136) [sync] Sync sender-side statement generation
yanze chen created IOTDB-4136: - Summary: [sync] Sync sender-side statement generation Key: IOTDB-4136 URL: https://issues.apache.org/jira/browse/IOTDB-4136 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: Sixing Wu Parsing the SQL related to Pipe operation and PipeSink operation（https://apache-iotdb.feishu.cn/docs/doccnJ6PdRBRDjlbgU8R3MoQdqf#2BNZww） There are at least five types of statements need to be generated: * ShowPipeSinkStatment * ShowPipeSinkTypeStatement * ShowPipeStatament * OperatePipeSinkStatement * OperatePipeStatement -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4137) [docs] User guide of new standalone sync
yanze chen created IOTDB-4137: - Summary: [docs] User guide of new standalone sync Key: IOTDB-4137 URL: https://issues.apache.org/jira/browse/IOTDB-4137 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yanze chen In the new version of mpp framework, some sync module configurations and SQL statements have been changed. For example, pipe_server_port, start pipeserver, show pipeserver, etc. Therefore, update the user manual after completing the function implementation. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4179) Clear iotdb-thrift-sync in pom file
yanze chen created IOTDB-4179: - Summary: Clear iotdb-thrift-sync in pom file Key: IOTDB-4179 URL: https://issues.apache.org/jira/browse/IOTDB-4179 Project: Apache IoTDB Issue Type: Bug Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4228) [sync] PipeSink execution process in new standalone
yanze chen created IOTDB-4228: - Summary: [sync] PipeSink execution process in new standalone Key: IOTDB-4228 URL: https://issues.apache.org/jira/browse/IOTDB-4228 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4229) Rename TransportClient to SyncClient
yanze chen created IOTDB-4229: - Summary: Rename TransportClient to SyncClient Key: IOTDB-4229 URL: https://issues.apache.org/jira/browse/IOTDB-4229 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4234) implicit definition of token PIPESERVER in parser
yanze chen created IOTDB-4234: - Summary: implicit definition of token PIPESERVER in parser Key: IOTDB-4234 URL: https://issues.apache.org/jira/browse/IOTDB-4234 Project: Apache IoTDB Issue Type: Bug Reporter: yanze chen Assignee: yanze chen Attachments: WechatIMG548.jpeg -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4248) [sync sender] Remove pipeMessage and merge the function into pipeinfo
yanze chen created IOTDB-4248: - Summary: [sync sender] Remove pipeMessage and merge the function into pipeinfo Key: IOTDB-4248 URL: https://issues.apache.org/jira/browse/IOTDB-4248 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen PipeMessage is not a good design because it needs to record message to disk. To support SHOW PIPE, we only record message in memeory as enum *WARN/ERROR.* see [https://apache-iotdb.feishu.cn/docs/doccnJ6PdRBRDjlbgU8R3MoQdqf#N9quKb] for more details. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4250) [sync sender] Support multiple pipes running at the same time
yanze chen created IOTDB-4250: - Summary: [sync sender] Support multiple pipes running at the same time Key: IOTDB-4250 URL: https://issues.apache.org/jira/browse/IOTDB-4250 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4270) [sync sender] Support parallel trasnport for each pipe
yanze chen created IOTDB-4270: - Summary: [sync sender] Support parallel trasnport for each pipe Key: IOTDB-4270 URL: https://issues.apache.org/jira/browse/IOTDB-4270 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4293) BufferedPipeDataQueue supports discontinuous serialNumber
yanze chen created IOTDB-4293: - Summary: BufferedPipeDataQueue supports discontinuous serialNumber Key: IOTDB-4293 URL: https://issues.apache.org/jira/browse/IOTDB-4293 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yusicheng * When serialNum is not discontinuous (e.g. 1, 2, 5, 8, 10), the correctness of the interface in PipeDataQueue should be ensured. * Add UT for discontinuous serialNumber case. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4320) [WAL] Recover insert row with null value throw NPE
yanze chen created IOTDB-4320:
-
Summary: [WAL] Recover insert row with null value throw NPE
Key: IOTDB-4320
URL: https://issues.apache.org/jira/browse/IOTDB-4320
Project: Apache IoTDB
Issue Type: Bug
Affects Versions: 0.13.2
Reporter: yanze chen
Assignee: yanze chen
execute:
{code:java}
create ALIGNED timeseries root.sg1(isa boolean encoding=RLE,isb boolean
encoding=RLE);
insert into root.sg1(timestamp,isa,isb) aligned VALUES (NOW(),False,null);
{code}
Restart IOTDB and report:
{code:java}
2022-09-02 14:56:14,415 [pool-10-IoTDB-Recovery-Thread-Pool-1] ERROR
o.a.i.d.w.i.SingleFileLogReader:108 - Cannot read more PhysicalPlans from 0,
successfully read index is
/var/lib/iotdb/data/wal/root.ceshi.p1.s_1/0-1662101750410-1-0-0.tsfile/wal1.
The reason is
java.nio.BufferUnderflowException: null
at java.base/java.nio.Buffer.nextGetIndex(Buffer.java:647)
at java.base/java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:388)
at
org.apache.iotdb.tsfile.utils.ReadWriteIOUtils.readInt(ReadWriteIOUtils.java:519)
at
org.apache.iotdb.db.qp.physical.crud.InsertRowPlan.fillValues(InsertRowPlan.java:459)
at
org.apache.iotdb.db.qp.physical.crud.InsertRowPlan.deserializeMeasurementsAndValues(InsertRowPlan.java:533)
at
org.apache.iotdb.db.qp.physical.crud.InsertRowPlan.deserialize(InsertRowPlan.java:518)
at
org.apache.iotdb.db.qp.physical.PhysicalPlan$Factory.create(PhysicalPlan.java:497)
at
org.apache.iotdb.db.writelog.io.BatchLogReader.readLogs(BatchLogReader.java:55)
at
org.apache.iotdb.db.writelog.io.BatchLogReader.(BatchLogReader.java:47)
at
org.apache.iotdb.db.writelog.io.SingleFileLogReader.hasNext(SingleFileLogReader.java:100)
at
org.apache.iotdb.db.writelog.io.MultiFileLogReader.hasNext(MultiFileLogReader.java:57)
at
org.apache.iotdb.db.writelog.recover.LogReplayer.replayLogs(LogReplayer.java:105)
at
org.apache.iotdb.db.writelog.recover.TsFileRecoverPerformer.redoLogs(TsFileRecoverPerformer.java:290)
at
org.apache.iotdb.db.writelog.recover.TsFileRecoverPerformer.recover(TsFileRecoverPerformer.java:143)
at
org.apache.iotdb.db.engine.storagegroup.VirtualStorageGroupProcessor.recoverTsFiles(VirtualStorageGroupProcessor.java:786)
at
org.apache.iotdb.db.engine.storagegroup.VirtualStorageGroupProcessor.recover(VirtualStorageGroupProcessor.java:527)
at
org.apache.iotdb.db.engine.storagegroup.VirtualStorageGroupProcessor.(VirtualStorageGroupProcessor.java:404)
at
org.apache.iotdb.db.engine.StorageEngine.buildNewStorageGroupProcessor(StorageEngine.java:707)
at
org.apache.iotdb.db.engine.storagegroup.virtualSg.StorageGroupManager.lambda$asyncRecover$0(StorageGroupManager.java:244)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IOTDB-4352) Fix PatialPath#overlapWith PartialPath contains MULTI_LEVEL_PATH_WILDCARD
yanze chen created IOTDB-4352: - Summary: Fix PatialPath#overlapWith PartialPath contains MULTI_LEVEL_PATH_WILDCARD Key: IOTDB-4352 URL: https://issues.apache.org/jira/browse/IOTDB-4352 Project: Apache IoTDB Issue Type: Bug Reporter: yanze chen Assignee: yanze chen PatialPath#overlapWith should not return true directly if PartialPath contains MULTI_LEVEL_PATH_WILDCARD. For example, root.**.s1 is not overlap with root.**.s2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4391) Apply PatternTreeMap to parse .mods file
yanze chen created IOTDB-4391: - Summary: Apply PatternTreeMap to parse .mods file Key: IOTDB-4391 URL: https://issues.apache.org/jira/browse/IOTDB-4391 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4393) [docs] Detailed design of distributed SYNC
yanze chen created IOTDB-4393: - Summary: [docs] Detailed design of distributed SYNC Key: IOTDB-4393 URL: https://issues.apache.org/jira/browse/IOTDB-4393 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen Detail design of distributed SYNC, mainly about ConfigNode * Interface * Interaction between DataNode and ConfigNode * Procedure state for sync https://apache-iotdb.feishu.cn/docx/doxcnpfFAOfJJGj6qmDM4VPEDNf -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4453) Fix the overflow of compareTo in TimeRange
yanze chen created IOTDB-4453: - Summary: Fix the overflow of compareTo in TimeRange Key: IOTDB-4453 URL: https://issues.apache.org/jira/browse/IOTDB-4453 Project: Apache IoTDB Issue Type: Bug Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4456) [sync] Sync cluster framework in ConfigNode
yanze chen created IOTDB-4456: - Summary: [sync] Sync cluster framework in ConfigNode Key: IOTDB-4456 URL: https://issues.apache.org/jira/browse/IOTDB-4456 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4459) [sync] PipeSink operation in cluster version
yanze chen created IOTDB-4459: - Summary: [sync] PipeSink operation in cluster version Key: IOTDB-4459 URL: https://issues.apache.org/jira/browse/IOTDB-4459 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4488) Implement ClusterSyncInfo snapshot by SyncLogWritter and SyncLogReader
yanze chen created IOTDB-4488: - Summary: Implement ClusterSyncInfo snapshot by SyncLogWritter and SyncLogReader Key: IOTDB-4488 URL: https://issues.apache.org/jira/browse/IOTDB-4488 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4528) [sync] Pipe operation in cluster version
yanze chen created IOTDB-4528: - Summary: [sync] Pipe operation in cluster version Key: IOTDB-4528 URL: https://issues.apache.org/jira/browse/IOTDB-4528 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-4402) Avoid multiple IoTDB processes accessing same data dir
[ https://issues.apache.org/jira/browse/IOTDB-4402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17610549#comment-17610549 ] yanze chen commented on IOTDB-4402: --- https://stackoverflow.com/questions/35842/how-can-a-java-program-get-its-own-process-id > Avoid multiple IoTDB processes accessing same data dir > --- > > Key: IOTDB-4402 > URL: https://issues.apache.org/jira/browse/IOTDB-4402 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Haonan Hou >Assignee: yanze chen >Priority: Major > > If multiple IoTDB processes operate the same data dirs, the data file may be > damaged. Need a method to protect the data dir. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4562) Optimize ITimeIndex#getDevices()
yanze chen created IOTDB-4562: - Summary: Optimize ITimeIndex#getDevices() Key: IOTDB-4562 URL: https://issues.apache.org/jira/browse/IOTDB-4562 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4570) Move SerializationRunTimeException to node-commons
yanze chen created IOTDB-4570: - Summary: Move SerializationRunTimeException to node-commons Key: IOTDB-4570 URL: https://issues.apache.org/jira/browse/IOTDB-4570 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4628) [sync] Do not store pipe infomation after drop
yanze chen created IOTDB-4628: - Summary: [sync] Do not store pipe infomation after drop Key: IOTDB-4628 URL: https://issues.apache.org/jira/browse/IOTDB-4628 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen In old version, pipe infomation will be stored even if pipe has been dropped. There are two advantages to not storing pipe infomation after drop can. # Support rollback logic in procedure better. If failed to create pipe, drop directlt. # Other modules(e.g.Trigger) do the same thing. Doing so can keep the semantics consistent in IoTDB. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-4022) Adjust the storage_group_report_threshold according to memory for writing
[ https://issues.apache.org/jira/browse/IOTDB-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616878#comment-17616878 ] yanze chen commented on IOTDB-4022: --- 更新测试： 吞吐(point/s)，写入内存分配=8980481489 || ||aligned_seq_config.properties||seq_config.properties||unseq_config.properties||aligned_unseq_config.properties|| |storage_group_report_threshold=16777216|8912483.19|17433312.42/ 16564385.56|3345125.65 |半个小时只跑了10%| |write_memory_vriation_report_proportion=0.001|13742844.78|16521936.57 /13984814.52|5082498.77 | | 吞吐(point/s)，总内存分配=100M || ||seq_config.properties|| |storage_group_report_threshold=16777216|写入一部分后OOM| |write_memory_vriation_report_proportion=0.001|写入失败，reject| > Adjust the storage_group_report_threshold according to memory for writing > - > > Key: IOTDB-4022 > URL: https://issues.apache.org/jira/browse/IOTDB-4022 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Jialin Qiao >Assignee: yanze chen >Priority: Major > Labels: pull-request-available > Attachments: 完整实验结果.md > > > Currently, we have a parameter called storage_group_report_threshold, which > means when a storage group report its memory usage, it should accumulate a > memory change of this number. > If this parameter is too small, may report frequently, which may decrease the > insertion performance. > If this parameter is too large, may report not in time, which may cause OOM. > > We need to change this parameter according to the memory allocated for > writing. > > (1) remove storage_group_report_threshold > (2) Add a new parameter write_memory_vriation_report_proportion > > To rel/0.13 and master -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4676) [sync] Support sync DeleteTimeseries
yanze chen created IOTDB-4676: - Summary: [sync] Support sync DeleteTimeseries Key: IOTDB-4676 URL: https://issues.apache.org/jira/browse/IOTDB-4676 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yusicheng * Support collect DeleteTimeseries in sender * Add new PipeData type * Support load DeleteTimeseries in receiver -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4677) [sync] Support sync DeleteData
yanze chen created IOTDB-4677: - Summary: [sync] Support sync DeleteData Key: IOTDB-4677 URL: https://issues.apache.org/jira/browse/IOTDB-4677 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yusicheng * Support collect DeleteData in sender * Add new PipeData type * Support load DeleteData in receiver -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4678) [sync] Sync DataNode start-up process
yanze chen created IOTDB-4678: - Summary: [sync] Sync DataNode start-up process Key: IOTDB-4678 URL: https://issues.apache.org/jira/browse/IOTDB-4678 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4692) SchemaFile adaptation for cluster metadata validation
yanze chen created IOTDB-4692: - Summary: SchemaFile adaptation for cluster metadata validation Key: IOTDB-4692 URL: https://issues.apache.org/jira/browse/IOTDB-4692 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yanze chen Support these two interface in SchemaRegionSchemaFileImpl: # checkMeasurementExistence() # fetchSchema() without template -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4714) [sync] Rename TPipeInfo to TCreatePipeReq
yanze chen created IOTDB-4714: - Summary: [sync] Rename TPipeInfo to TCreatePipeReq Key: IOTDB-4714 URL: https://issues.apache.org/jira/browse/IOTDB-4714 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4719) [SchemaFile] Adaptation for cluster delete timeseries
yanze chen created IOTDB-4719: - Summary: [SchemaFile] Adaptation for cluster delete timeseries Key: IOTDB-4719 URL: https://issues.apache.org/jira/browse/IOTDB-4719 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yanze chen Support 3 interface in SchemaRegionSchemaFileImpl: # constructSchemaBlackList # rollbackSchemaBlackList # deleteTimeseriesInBlackList -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-4628) [sync] Do not store pipe infomation after drop
[ https://issues.apache.org/jira/browse/IOTDB-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623479#comment-17623479 ] yanze chen commented on IOTDB-4628: --- https://github.com/apache/iotdb/pull/7581 > [sync] Do not store pipe infomation after drop > -- > > Key: IOTDB-4628 > URL: https://issues.apache.org/jira/browse/IOTDB-4628 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: yanze chen >Assignee: yanze chen >Priority: Major > > In old version, pipe infomation will be stored even if pipe has been dropped. > There are two advantages to not storing pipe infomation after drop can. > # Support rollback logic in procedure better. If failed to create pipe, drop > directlt. > # Other modules(e.g.Trigger) do the same thing. Doing so can keep the > semantics consistent in IoTDB. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4743) [sync] OperatePipeProcedure support rollback
yanze chen created IOTDB-4743: - Summary: [sync] OperatePipeProcedure support rollback Key: IOTDB-4743 URL: https://issues.apache.org/jira/browse/IOTDB-4743 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yanze chen OperatePipeProcedure support rollback -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4799) [sync] Print more detailed pipe information in show pipe
yanze chen created IOTDB-4799: - Summary: [sync] Print more detailed pipe information in show pipe Key: IOTDB-4799 URL: https://issues.apache.org/jira/browse/IOTDB-4799 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yanze chen * Add a column attributes to show pipe attributes * For receiver, show connection of sender -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4808) Rewrite SyncTransportTest in new IT framework
yanze chen created IOTDB-4808: - Summary: Rewrite SyncTransportTest in new IT framework Key: IOTDB-4808 URL: https://issues.apache.org/jira/browse/IOTDB-4808 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4827) [SchemaFile] Adaptation for cluster template
yanze chen created IOTDB-4827: - Summary: [SchemaFile] Adaptation for cluster template Key: IOTDB-4827 URL: https://issues.apache.org/jira/browse/IOTDB-4827 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yanze chen # activateSchemaTemplate # getPathsUsingTemplate # constructSchemaBlackListWithTemplate # rollbackSchemaBlackListWithTemplate # deactivateTemplateInBlackList # countPathsUsingTemplate # getAllTimeseriesCount, fetchSchema andshowTimeseriesWithoutIndex with template -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4840) [schema] Fix MLog writing logic about template
yanze chen created IOTDB-4840: - Summary: [schema] Fix MLog writing logic about template Key: IOTDB-4840 URL: https://issues.apache.org/jira/browse/IOTDB-4840 Project: Apache IoTDB Issue Type: Bug Reporter: yanze chen Assignee: yanze chen Attachments: image-2022-11-02-20-52-49-392.png MLog entry should be splitted into actual fullPath and templateId !image-2022-11-02-20-52-49-392.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4854) [UT] Migrate SchemaBasicTest to SchemaRegionBasicTest
yanze chen created IOTDB-4854: - Summary: [UT] Migrate SchemaBasicTest to SchemaRegionBasicTest Key: IOTDB-4854 URL: https://issues.apache.org/jira/browse/IOTDB-4854 Project: Apache IoTDB Issue Type: Bug Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-4209) [ schema recovery ] Aligned timeseries recovery is slow
[
https://issues.apache.org/jira/browse/IOTDB-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yanze chen reassigned IOTDB-4209:
-
Assignee: yanze chen (was: ZhaoXin)
> [ schema recovery ] Aligned timeseries recovery is slow
> ---
>
> Key: IOTDB-4209
> URL: https://issues.apache.org/jira/browse/IOTDB-4209
> Project: Apache IoTDB
> Issue Type: Improvement
> Components: Core/Schema Manager
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: yanze chen
>Priority: Major
>
> 单机，sf_tree_seg，0819_207abf6
> bm创建对齐序列耗时650.73 second
> 恢复这些序列耗时{color:#DE350B}*1716436ms*{color}， 需优化对齐序列的恢复性能。
> bm运行时间19分钟：
> 2022-08-22 09:56:01 到 2022-08-22 10:15:42
> {color:#00875A}*Create schema cost 650.73 second*{color}
> Test elapsed time (not include schema creation): 308.10 second
> iotdb重启恢复：
> 2022-08-22 13:19:19,195 [main] INFO o.a.i.db.service.IoTDB:140 - recover the
> schema...
> 2022-08-22 13:47:55,631 [main] INFO o.a.i.db.service.IoTDB:234 - spend
> {color:#DE350B}*1716436ms*{color} to recover schema.
> log_all.log 耗时38分钟
> 2022-08-22 13:19:18 到2022-08-22 13:57:52,856 [main] INFO
> o.a.i.db.service.IoTDB:198 - Congratulation, IoTDB is set up successfully.
> Now, enjoy yourself!
> 复现流程：
> 1. 192.168.10.64 72C256G
> MAX_HEAP_SIZE="192G"
> MAX_DIRECT_MEMORY_SIZE="32G"
> mlog_buffer_size=10485760
> schema_engine_mode=Schema_File
> 2. benchmark运行附件中的脚本
> benchmark在本机路径/data/liuzhen_test/weektest/benchmark_tool_v13 （0620的benchmark）
> 主要配置
> DEVICE_NUMBER=500
> SENSOR_NUMBER=20
> CLIENT_NUMBER=20
> GROUP_NUMBER=100
> VECTOR=true
> LOOP=1
> BATCH_SIZE_PER_WRITE=1
> OPERATION_PROPORTION=1:0:0:0:0:0:0:0:0:0:0
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IOTDB-4908) Clear deprecayed code related to sync in old standalone
yanze chen created IOTDB-4908: - Summary: Clear deprecayed code related to sync in old standalone Key: IOTDB-4908 URL: https://issues.apache.org/jira/browse/IOTDB-4908 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4941) Prevent compatibility problems in PipeData
yanze chen created IOTDB-4941: - Summary: Prevent compatibility problems in PipeData Key: IOTDB-4941 URL: https://issues.apache.org/jira/browse/IOTDB-4941 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen Prevent compatibility problems in PipeData beacause of ordinal of PipeDataType -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4960) [SchemaFile] Query large amounts of metadata throw ClosedChannelException
yanze chen created IOTDB-4960:
-
Summary: [SchemaFile] Query large amounts of metadata throw
ClosedChannelException
Key: IOTDB-4960
URL: https://issues.apache.org/jira/browse/IOTDB-4960
Project: Apache IoTDB
Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen
Attachments: image-2022-11-16-15-29-18-436.png
reproduce:
1C1D Schema_File mode 128G for DataNode
run benchmark with config:
*DEVICE_NUMBER*=5000
*SENSOR_NUMBER*=2
*CLIENT_NUMBER*=5
*GROUP_NUMBER*=10
*BATCH_SIZE_PER_WRITE*=0
{*}LOOP{*}=1
execute SQL:
count timeseries root.**
!image-2022-11-16-15-29-18-436.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IOTDB-4960) [SchemaFile] Query large amounts of metadata throw ClosedChannelException
[
https://issues.apache.org/jira/browse/IOTDB-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17634708#comment-17634708
]
yanze chen commented on IOTDB-4960:
---
原因可能是mpp框架查询超时，interrupt掉了正在read的channel，导致channel关闭
!image-2022-11-16-16-01-50-022.png|width=827,height=407!
> [SchemaFile] Query large amounts of metadata throw ClosedChannelException
> -
>
> Key: IOTDB-4960
> URL: https://issues.apache.org/jira/browse/IOTDB-4960
> Project: Apache IoTDB
> Issue Type: Bug
>Reporter: yanze chen
>Assignee: yanze chen
>Priority: Major
> Attachments: image-2022-11-16-15-29-18-436.png,
> image-2022-11-16-16-01-50-022.png
>
>
> reproduce:
> 1C1D Schema_File mode 128G for DataNode
>
> run benchmark with config:
> {*}DEVICE_NUMBER{*}=5000
> {*}SENSOR_NUMBER{*}=2
> {*}CLIENT_NUMBER{*}=5
> {*}GROUP_NUMBER{*}=10
> {*}BATCH_SIZE_PER_WRITE{*}=0
> {*}LOOP{*}=1
> {*}VECTOR{*}=true
>
> execute SQL:
> count timeseries root.**
>
> !image-2022-11-16-15-29-18-436.png|width=881,height=596!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IOTDB-4970) [sync] Enhanced sync safety management
yanze chen created IOTDB-4970: - Summary: [sync] Enhanced sync safety management Key: IOTDB-4970 URL: https://issues.apache.org/jira/browse/IOTDB-4970 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen Before the sync sender can send data, a handshake with the receiver is required. The current handshake policy is to verify the sender's IP address based on the receiver's white list. However, the current sender's ip address is provided by the sender and may be falsified leading to security issues. # This ISSUE will get the ip address from the thrift connection. # The default whitelist is set to 0.0.0.0/32, which rejects all the sender's ip. # Some optimizations have been made to handle cases of handshake failure. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5021) Using Junit parameterized test to optimize schema IT framework
yanze chen created IOTDB-5021: - Summary: Using Junit parameterized test to optimize schema IT framework Key: IOTDB-5021 URL: https://issues.apache.org/jira/browse/IOTDB-5021 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen In order to make the tests cover Schema_File mode and Memory mode, we originally needed to create two classes for each IT, such as IoTDBMetadataFetchIT and IoTDBMetadataFetchSchemaFileIT, which would result in a large number of classes. This ISSUE introduces a JUnit parametric test approach. An AbstractSchemaIT abstract class is created for all schema test, and its subclasses will automatically run tests in both schema engine modes. reference: [https://blog.csdn.net/w605283073/article/details/80051229] [https://stackoverflow.com/questions/27745691/how-to-combine-runwith-with-runwithparameterized-class] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5106) Parse PathPattern to Deterministic Finite Automate
yanze chen created IOTDB-5106: - Summary: Parse PathPattern to Deterministic Finite Automate Key: IOTDB-5106 URL: https://issues.apache.org/jira/browse/IOTDB-5106 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen https://apache-iotdb.feishu.cn/docx/DjrAdDM3woXcn1x5Phlcyveznqe -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-5106) Parse PathPattern to Deterministic Finite Automate
[ https://issues.apache.org/jira/browse/IOTDB-5106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanze chen reassigned IOTDB-5106: - Assignee: yanze chen > Parse PathPattern to Deterministic Finite Automate > -- > > Key: IOTDB-5106 > URL: https://issues.apache.org/jira/browse/IOTDB-5106 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: yanze chen >Assignee: yanze chen >Priority: Major > > https://apache-iotdb.feishu.cn/docx/DjrAdDM3woXcn1x5Phlcyveznqe -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5133) [0.13] Export schema tool
yanze chen created IOTDB-5133: - Summary: [0.13] Export schema tool Key: IOTDB-5133 URL: https://issues.apache.org/jira/browse/IOTDB-5133 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yanze chen Add an export schema function to export MTree and Tag from the current system into a non-deleted mlog.bin for loading mlog in a non-empty iotdb correctly. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5201) Extracting getConfig from the registerDataNode interface
yanze chen created IOTDB-5201: - Summary: Extracting getConfig from the registerDataNode interface Key: IOTDB-5201 URL: https://issues.apache.org/jira/browse/IOTDB-5201 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen Attachments: image-2022-12-13-20-59-57-380.png Extracting getConfig from the registerDataNode interface !image-2022-12-13-20-59-57-380.png|width=591,height=308! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5223) Check data directories cross-disk under RatisConsensus
yanze chen created IOTDB-5223: - Summary: Check data directories cross-disk under RatisConsensus Key: IOTDB-5223 URL: https://issues.apache.org/jira/browse/IOTDB-5223 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen Configuring the data directories as cross-disk directories is not supported under RatisConsensus(it will be supported in a later version) _https://stackoverflow.com/questions/56712909/get-a-folders-mount-point-using-java_ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5235) [SchemaFile] support props
yanze chen created IOTDB-5235: - Summary: [SchemaFile] support props Key: IOTDB-5235 URL: https://issues.apache.org/jira/browse/IOTDB-5235 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yanze chen SchemaFile need to support props. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5238) UT related to alias and tags and attributes
yanze chen created IOTDB-5238: - Summary: UT related to alias and tags and attributes Key: IOTDB-5238 URL: https://issues.apache.org/jira/browse/IOTDB-5238 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen UT related to alias and tags and attributes -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5239) UT related to delete timeseries
yanze chen created IOTDB-5239: - Summary: UT related to delete timeseries Key: IOTDB-5239 URL: https://issues.apache.org/jira/browse/IOTDB-5239 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen UT related to delete timeseries -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5242) UT related to MetaUtils
yanze chen created IOTDB-5242: - Summary: UT related to MetaUtils Key: IOTDB-5242 URL: https://issues.apache.org/jira/browse/IOTDB-5242 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5247) [SchemaFile] Drop tags or attributes throw NPE
yanze chen created IOTDB-5247:
-
Summary: [SchemaFile] Drop tags or attributes throw NPE
Key: IOTDB-5247
URL: https://issues.apache.org/jira/browse/IOTDB-5247
Project: Apache IoTDB
Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen
Reproduce:
{code:java}
// SchemaRegionAliasAndTagTest.java
@Test
public void testDropTagsOrAttributes() {
try {
prepareTimeseries();
Set keySet = new HashSet<>(Arrays.asList("tag1", "tag2", "attr1",
"attr2"));
List fullPaths =
Arrays.asList(
"root.sg.wf01.wt01.v1.s1",
"root.sg.wf01.wt01.v1.s2",
"root.sg.wf01.aligned_device1.s1",
"root.sg.wf01.aligned_device1.s2",
"root.sg.wf01.aligned_device2.s1",
"root.sg.wf01.aligned_device2.s2");
for (String fullPath : fullPaths) {
schemaRegion.dropTagsOrAttributes(keySet, new PartialPath(fullPath));
checkTags(fullPath, Collections.emptyMap());
checkAttributes(fullPath, Collections.emptyMap());
}
} catch (Exception e) {
logger.error(e.getMessage(), e);
Assert.fail(e.getMessage());
}
} {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IOTDB-5277) [schema] SchemaRegion throws NPE when loading snapshot
yanze chen created IOTDB-5277:
-
Summary: [schema] SchemaRegion throws NPE when loading snapshot
Key: IOTDB-5277
URL: https://issues.apache.org/jira/browse/IOTDB-5277
Project: Apache IoTDB
Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen
Attachments: 9b23c97eca9eeb1adc11571607baa3f6.zip
reproduce: Launch 1C1D using the attached data directory
{code:java}
1972 [main] ERROR o.a.iotdb.commons.ServerCommandLine - Failed to execute
system command
java.util.concurrent.CompletionException: java.lang.NullPointerException:
Cannot invoke
"org.apache.iotdb.db.metadata.mnode.IMNode.getAsStorageGroupMNode()" because
the return value of
"org.apache.iotdb.db.metadata.mtree.store.MemMTreeStore.getRoot()" is null
at
java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332)
at
java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:347)
at
java.base/java.util.concurrent.CompletableFuture$BiRelay.tryFire(CompletableFuture.java:1498)
at
java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
at
java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147)
at org.apache.ratis.util.ConcurrentUtils.accept(ConcurrentUtils.java:174)
at
org.apache.ratis.util.ConcurrentUtils.lambda$null$3(ConcurrentUtils.java:165)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.NullPointerException: Cannot invoke
"org.apache.iotdb.db.metadata.mnode.IMNode.getAsStorageGroupMNode()" because
the return value of
"org.apache.iotdb.db.metadata.mtree.store.MemMTreeStore.getRoot()" is null
at
org.apache.iotdb.db.metadata.mtree.MTreeBelowSGMemoryImpl.(MTreeBelowSGMemoryImpl.java:133)
at
org.apache.iotdb.db.metadata.mtree.MTreeBelowSGMemoryImpl.loadFromSnapshot(MTreeBelowSGMemoryImpl.java:173)
at
org.apache.iotdb.db.metadata.schemaregion.SchemaRegionMemoryImpl.loadSnapshot(SchemaRegionMemoryImpl.java:457)
at
org.apache.iotdb.db.consensus.statemachine.SchemaRegionStateMachine.loadSnapshot(SchemaRegionStateMachine.java:67)
at
org.apache.iotdb.consensus.ratis.ApplicationStateMachineProxy.loadSnapshot(ApplicationStateMachineProxy.java:257)
at org.a {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IOTDB-5329) Exception handler for AbstractTreeVisitor
yanze chen created IOTDB-5329: - Summary: Exception handler for AbstractTreeVisitor Key: IOTDB-5329 URL: https://issues.apache.org/jira/browse/IOTDB-5329 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen Add exception handler for AbstractTreeVisitor -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5342) Separate filtering and business logic in AbstractTreeVisitor
yanze chen created IOTDB-5342: - Summary: Separate filtering and business logic in AbstractTreeVisitor Key: IOTDB-5342 URL: https://issues.apache.org/jira/browse/IOTDB-5342 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen Separate filtering and business logic in AbstractTreeVisitor -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5359) Use decorator pattern to refactor SchemaTreeVisitorWithLimitOffset
yanze chen created IOTDB-5359: - Summary: Use decorator pattern to refactor SchemaTreeVisitorWithLimitOffset Key: IOTDB-5359 URL: https://issues.apache.org/jira/browse/IOTDB-5359 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen Use decorator pattern to refactor SchemaTreeVisitorWithLimitOffset -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5360) Release node and iterator in AbstractTreeVisitor
yanze chen created IOTDB-5360: - Summary: Release node and iterator in AbstractTreeVisitor Key: IOTDB-5360 URL: https://issues.apache.org/jira/browse/IOTDB-5360 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen To support pin and unpin in SchemaFile mode, AbstractTreeVisitor should release node and iterator after using. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-5342) Separate filtering and business logic in AbstractTreeVisitor
[ https://issues.apache.org/jira/browse/IOTDB-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655287#comment-17655287 ] yanze chen commented on IOTDB-5342: --- https://github.com/apache/iotdb/pull/8721 > Separate filtering and business logic in AbstractTreeVisitor > > > Key: IOTDB-5342 > URL: https://issues.apache.org/jira/browse/IOTDB-5342 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: yanze chen >Assignee: yanze chen >Priority: Major > Labels: pull-request-available > > Separate filtering and business logic in AbstractTreeVisitor -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-5359) Use decorator pattern to refactor SchemaTreeVisitorWithLimitOffset
[ https://issues.apache.org/jira/browse/IOTDB-5359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655289#comment-17655289 ] yanze chen commented on IOTDB-5359: --- https://github.com/apache/iotdb/pull/8721 > Use decorator pattern to refactor SchemaTreeVisitorWithLimitOffset > -- > > Key: IOTDB-5359 > URL: https://issues.apache.org/jira/browse/IOTDB-5359 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: yanze chen >Assignee: yanze chen >Priority: Major > > Use decorator pattern to refactor SchemaTreeVisitorWithLimitOffset -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-5360) Release node and iterator in AbstractTreeVisitor
[ https://issues.apache.org/jira/browse/IOTDB-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655288#comment-17655288 ] yanze chen commented on IOTDB-5360: --- https://github.com/apache/iotdb/pull/8721 > Release node and iterator in AbstractTreeVisitor > > > Key: IOTDB-5360 > URL: https://issues.apache.org/jira/browse/IOTDB-5360 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: yanze chen >Assignee: yanze chen >Priority: Major > > To support pin and unpin in SchemaFile mode, AbstractTreeVisitor should > release node and iterator after using. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-5235) [SchemaFile] support props
[ https://issues.apache.org/jira/browse/IOTDB-5235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanze chen reassigned IOTDB-5235: - Assignee: ZhaoXin (was: yanze chen) > [SchemaFile] support props > -- > > Key: IOTDB-5235 > URL: https://issues.apache.org/jira/browse/IOTDB-5235 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: yanze chen >Assignee: ZhaoXin >Priority: Major > > SchemaFile need to support props. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-4960) [SchemaFile] Query large amounts of metadata throw ClosedChannelException
[
https://issues.apache.org/jira/browse/IOTDB-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yanze chen reassigned IOTDB-4960:
-
Assignee: ZhaoXin (was: yanze chen)
> [SchemaFile] Query large amounts of metadata throw ClosedChannelException
> -
>
> Key: IOTDB-4960
> URL: https://issues.apache.org/jira/browse/IOTDB-4960
> Project: Apache IoTDB
> Issue Type: Bug
>Reporter: yanze chen
>Assignee: ZhaoXin
>Priority: Major
> Attachments: image-2022-11-16-15-29-18-436.png,
> image-2022-11-16-16-01-50-022.png
>
>
> reproduce:
> 1C1D Schema_File mode 128G for DataNode
>
> run benchmark with config:
> {*}DEVICE_NUMBER{*}=5000
> {*}SENSOR_NUMBER{*}=2
> {*}CLIENT_NUMBER{*}=5
> {*}GROUP_NUMBER{*}=10
> {*}BATCH_SIZE_PER_WRITE{*}=0
> {*}LOOP{*}=1
> {*}VECTOR{*}=true
>
> execute SQL:
> count timeseries root.**
>
> !image-2022-11-16-15-29-18-436.png|width=881,height=596!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IOTDB-5391) Implement SchemaReader using iterative Traverser
yanze chen created IOTDB-5391: - Summary: Implement SchemaReader using iterative Traverser Key: IOTDB-5391 URL: https://issues.apache.org/jira/browse/IOTDB-5391 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen The current SchemaReader uses the result set returned from the original SchemaRegion to generate an iterator; however, the new Traverser supports the iterator pattern, allowing the SchemaReader to be implemented using an iterative Traverser, avoiding the overhead of repeated traversals. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5396) Iterative interface for query about TagManager
yanze chen created IOTDB-5396: - Summary: Iterative interface for query about TagManager Key: IOTDB-5396 URL: https://issues.apache.org/jira/browse/IOTDB-5396 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen The current query interface relating to TagManager returns a collection of type list. For better adaptation to SchemaReader, iterator should be returned. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5398) Handle failure in schema query if there is exception occurs during the iteration
yanze chen created IOTDB-5398: - Summary: Handle failure in schema query if there is exception occurs during the iteration Key: IOTDB-5398 URL: https://issues.apache.org/jira/browse/IOTDB-5398 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen Handle failure in schema query if there is exception occurs during the iteration -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5413) ReadWriteLock for CachedMTreeStore
yanze chen created IOTDB-5413: - Summary: ReadWriteLock for CachedMTreeStore Key: IOTDB-5413 URL: https://issues.apache.org/jira/browse/IOTDB-5413 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen https://apache-iotdb.feishu.cn/docx/WwdAd6SYLofsAHxEJeRcDuWBnPh -- This message was sent by Atlassian Jira (v8.20.10#820010)
