[ 
https://issues.apache.org/jira/browse/IOTDB-3045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

刘珍 reopened IOTDB-3045:
-----------------------

master_0508_be385cd
100sg,500dev,20w序列/dev,共1亿序列,每个序列有10个点的数据。
每个dev delete 101个ts,重启恢复,部分dev的序列数是20万,期望是199899。

机器 :192.168.10.68
 MAX_HEAP_SIZE="192G"
 MAX_DIRECT_MEMORY_SIZE="32G"
schema_engine_mode=Schema_File

benchmark配置:
GROUP_NUMBER=100
DEVICE_NUMBER=500
SENSOR_NUMBER=200000
CLIENT_NUMBER=50
BATCH_SIZE_PER_WRITE=1
LOOP=10

脚本${iotdb_dir}下:
先获取所有dev的信息
cat get_dev_name.sh
#!/bin/bash
./sbin/start-cli.sh  -e "show devices;" |grep root|awk -F '|' '{print $2}' > 
dev_name.txt

再执行delete ts(10用户并发)
 cat get_dev_name.sh
#!/bin/bash
./sbin/start-cli.sh  -e "show devices;" |grep root|awk -F '|' '{print $2}' > 
dev_name.txt

liuzhen@fit-68:/data/liuzhen_test/master_0508_be385cd$ cat del_ts.sh
#!/bin/bash
function delete_ts()
{
   del_ts=$1
   dev_name=$2
   exp_count=$3
   count_ts ${exp_count} ${dev_name}
   ./sbin/start-cli.sh -e "delete timeseries ${del_ts};"
   let exp_count--
   count_ts ${exp_count} ${dev_name}
}

function count_ts()
{
    exp_c_ts=$1
    dev_name=$2
    cur_c_ts=`./sbin/start-cli.sh -e "count timeseries ${dev_name};"|sed -n 
"4,4p"|awk -F '|' '{print $2}'`
    if [ "${cur_c_ts}" -eq "${exp_c_ts}" ];then
        echo "ok"
    else
        echo "fail. ${dev_name}  ${exp_c_ts} ${cur_c_ts}" >>./fail.log
    fi
}

function execute()
{
    in_file=$1
    i=$2
    exp_count=$3
    n=0
    cat ${in_file} | while read line
    do
       del_ts="${line}.s_${i}"
       dev_name="${line}.*";
       delete_ts ${del_ts} ${dev_name} ${exp_count} &
       let i++
       let n++
       b=$(( $n % 10 ))
       if [ "$b" -eq "0" ];then
          wait
       fi
    done
    wait
}

function execute_ts_maxID()
{
    in_file=$1
    maxID=$2
    exp_count=$3
    n=0
    cat ${in_file} | while read line
    do
       del_ts="${line}.s_${maxID}"
       dev_name="${line}.*";
       delete_ts ${del_ts} ${dev_name} ${exp_count} &
       let n++
       b=$(( $n % 10 ))
       if [ "$b" -eq "0" ];then
          wait
       fi
    done
    wait
}


# 循环几轮
count_ts_base=200000

for loop in {0..99}
do
    idx=$(( $loop * 2000 ))
    execute "dev_name.txt" ${idx} ${count_ts_base}
    let count_ts_base--
done

idx=199999
execute_ts_maxID "dev_name.txt" ${idx} ${count_ts_base}


执行完后,再查询一遍所有dev下的count timeseries , 均是199899。表现正确。
停iotdb。
备份数据。
启动iotdb,查询一遍所有dev下的count timeseries ,部分结果是200000,delete成功的ts又查询到了。


> [ schema recovery ] The query result contains some timeseries that have been 
> deleted
> ------------------------------------------------------------------------------------
>
>                 Key: IOTDB-3045
>                 URL: https://issues.apache.org/jira/browse/IOTDB-3045
>             Project: Apache IoTDB
>          Issue Type: Bug
>          Components: Core/Schema Manager
>    Affects Versions: 0.14.0-SNAPSHOT
>            Reporter: 刘珍
>            Assignee: Jialin Qiao
>            Priority: Major
>         Attachments: config.properties, d1.log
>
>
> master_0428_ca4f3cf
> 问题描述:
> 50 sg,100 dev,200000 sensors/dev, 共2000万序列。
> 100个设备名称记录在文件中,
> 遍历文件,10个用户并发 delete ts,1个设备上delete 1条序列。
> 重复10次,也就是这100个dev,每个dev共delete 10个ts。
> delete 后,执行count timeseries dev.* ;结果符合预期。
> 备份数据。
> {color:#DE350B}重启iotdb,恢复元数据后,再查询 100个dev的序列数,有14个设备的序列数都是20万,
> 期望是199990,delete的ts,又查询到了。{color}
> 详细测试流程:
> 1. 私有云 172.20.70.2(iotdb)/16(bm)   8C32G
> iotdb参数
> MAX_HEAP_SIZE="20G"
> MAX_DIRECT_MEMORY_SIZE="6G"
> schema_engine_mode=Schema_File
> 2. benchmark 配置见附件
> 3. 获取dev 名称,记录到文件dev_name.txt
>  cat get_dev_name.sh
> #!/bin/bash
> ./sbin/start-cli.sh  -e "show devices;" |grep root|awk -F '|' '{print $2}' > 
> dev_name.txt
> 4. 并发delete
> cat del_1_ts.sh
> #!/bin/bash
> function delete_ts()
> {
>    del_ts=$1
>    dev_name=$2
>    exp_count=$3
>    count_ts ${exp_count} ${dev_name}
>    ./sbin/start-cli.sh -e "delete timeseries ${del_ts};"
>    let exp_count--
>    count_ts ${exp_count} ${dev_name}
> }
> function count_ts()
> {
>     exp_c_ts=$1
>     dev_name=$2
>     cur_c_ts=`./sbin/start-cli.sh -e "count timeseries ${dev_name};"|sed -n 
> "4,4p"|awk -F '|' '{print $2}'`
>     if [ "${cur_c_ts}" -eq "${exp_c_ts}" ];then
>         echo "ok"
>     else
>         echo "fail. ${dev_name}  ${exp_c_ts} ${cur_c_ts}" >>./fail.log
>     fi
> }
> function execute()
> {
>     in_file=$1
>     i=$2
>     exp_count=$3
>     n=0
>     cat ${in_file} | while read line
>     do
>        del_ts="${line}.s_${i}"
>        dev_name="${line}.*";
>        delete_ts ${del_ts} ${dev_name} ${exp_count} &
>        let i++
>        let n++
>        b=$(( $n % 10 ))
>        if [ "$b" -eq "0" ];then
>           wait
>        fi
>     done
>     wait
> }
> # 循环几轮
> count_ts_base=200000
> for loop in {1..10}
> do
>     execute "dev_name.txt" ${loop} ${count_ts_base}
>     let count_ts_base--
> done
> 并发delete脚本的日志文件见附件。
> 执行count_ts.sh,查看每个dev的count timeseries,是否都是199990。
> 有14个dev的结果是200000,不符合预期结果。



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

Reply via email to