Hi,
I convert figure to symbolic..
* when we flush 5(max_vm_num) times, the current level will do compaction
to the next level
1 1 1 1 1
| / / / /
5
* when we flush all 11 times, the compaction procedure is
1 1 1 1 1 1 1 1 1 1 1
| / / / / | / / / /
5 5
* if we close the tsfile, the
Hey Jialin, xinagdong,
very good question!
And I tend to agree with Xiangdong.
If the users do it that way it probably makes most sense for them.
The question I would ask is why "devices" hurt us (I know a bit about the
implementation of course but probably we have to adopt our datamodel also a
Hi,
max_vm_num means that the most number of vm files relation to a tsfile's level.
for example, we set max_vm_num=5 and we flush 11 times, then the compaction
procedure can be described as below:
* when we flush 5(max_vm_num)times, the current level will do compaction
to the next level
* when
Hi,
OK??I will do not use any rich-format
Suppose the parameter is 5. Then in level 2, will you merge 4 new VM files
to the bigger one, or merge 5 VM files?
I will merge 5 VM files to a bigger one in level 2.
-- --
??:
Hi, Xiangdong
You are right. Thank you for correction.
Xiangdong Huang 于2020年7月20日周一 下午3:29写道：
> Hi Xiangwei,
>
> > Besides, double quote is used as quote while single quote is for string
> literal.
>
> Are you meaning double quote is used as escape character..
>
> Best,
>
Hey Xiangwei,
that is an excellent suggestion (I discussed the same already with Xiangdong
some weeks ago).
We mix a bit between Strings and Paths (and even inside paths we use strings
and dynamic parsing too often).
So it would make sense to mee to follow your suggested approach with clear
Hi Lingzhe,
Suggest you give up your email client...
Or, do not use any rich-format in the mailing list.
Best,
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
445073309 于2020年7月20日周一 下午3:47写道：
> Hi,
>
>
> I convert figure to
Hi,
OK??I will do not use any rich-format
Suppose the parameter is 5. Then in level 2, will you merge 4 new VM files
to the bigger one, or merge 5 VM files?
I will merge 5 VM files to a bigger one in level 2.
----
??:
Hi,
> The question I would ask is why "devices" hurt us.
I'd like to introduce this a bit. For each storage group, we flush the memtable
into TsFiles one by one. For each TsFile, we maintain a temporal index on
device level in memory. Suppose there are 3 devices in one TsFile, the index is
Thanks fort he clear explanation, yes I remember that there were also reported
performance issues with that.
But to generalize the concept of a device all we would need is a tree strucutre
where each node has start time / end time for "everything" in the file.
Like in your example:
Root (1,
Hi Lingzhe,
>max_vm_num: indicates that a TsFileProcessor has at most the number of
virtual memory files
what does this mean? and how do I know what value is suitable? (For
example, if I set it as 1, is there any impact?)
Best,
---
Xiangdong Huang
School of
Hi
Recently, I find that some users create timeseries do not following the real
world semantic of device
E.g., a device has a sensor that collects some data in array format (int[3])
and some in long type.
Many users will create timeseries like this:
root.sg.device1.measurement1.int0
Hi,
Thanks Lingzhe, this feature could improve the query performance a lot.
max_vm_num limits the max number of vm in each level.
The max_vm_num is 10 by default and the max_merge_chunk_num_in_tsfile is 100
now.
Besides, I can't see you figures attached...
Thanks,
--
Jialin Qiao
Hi,
This is a quite good topic!
1. maybe we should hear more users opinions.
For me, I think emphasize the concept of "device" is good. We can even
expose the concept in our APIs.
2.
> A more efficient way is
> root.sg.device1.measurement1_int0
> root.sg.device1.measurement1_int1
>
Hi Xiangwei,
> Besides, double quote is used as quote while single quote is for string
literal.
Are you meaning double quote is used as escape character..
Best,
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Xiangwei Wei
Hi,
Did you attach some figures? The mailing list does not allow figures..
Suppose the parameter is 5. Then in level 2, will you merge 4 new VM files
to the bigger one, or merge 5 VM files?
Best,
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
Hi，
> I wonder whether we could index the file by its name. (naming the tsfile by
> date) E.g., we store each day's data in one file and name it as
> sg-2020-07-20.TsFile. Then, we do not need to maintain the index in memory,
> we just need to check whether the file exist in the queried
Hi,
> I wonder whether we could index the file by its name. (naming the tsfile
by date)
I think it is a good idea, but maybe not very easy to implement. If we can
organize the data like this, then it is very very regular and very easy to
access or delete expired data...
> we would need is a
18 matches
Mail list logo
