Re: (IOTDB-290) Bug about threadlocal field in TSServiceImpl.java

[田原]

Hi,

I have solved the issue. I create a new ThreadLocal field in TSServiceImpl 
class named queryId to generate unique queryId for each query in one session. 

ThreadLocal queryId = new ThreadLocal<>();

And also, the key of queryStatus and queryDataSets is changed to Long type 
which is actually the queryId.

The PR link is https://github.com/apache/incubator-iotdb/pull/522.


> -原始邮件-
> 发件人: "Yuan Tian (Jira)" 
> 发送时间: 2019-11-05 11:29:00 (星期二)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: [jira] [Created] (IOTDB-290) Bug about threadlocal field in 
> TSServiceImpl.java
> 
> Yuan Tian created IOTDB-290:
> ---
> 
>  Summary: Bug about threadlocal field in TSServiceImpl.java
>  Key: IOTDB-290
>  URL: https://issues.apache.org/jira/browse/IOTDB-290
>  Project: Apache IoTDB
>   Issue Type: Bug
> Reporter: Yuan Tian
> 
> 
> There are two fields named queryStatus and queryRet:
> {code:java}
> private ThreadLocal> queryStatus = new 
> ThreadLocal<>();
> private ThreadLocal> queryRet = new 
> ThreadLocal<>();
> {code}
> 
> They are used to save the PhysicalPlan and QueryDataSet of one query. 
> However, the keys of both are sql string to be executed, so if the client 
> execute the same sql in one connection, no matter whether the first execution 
> has finished, the second sql execution will clear the first one's 
> QueryDataSet.
> 
> 
> 
> --
> This message was sent by Atlassian Jira
> (v8.3.4#803005)

[jira] [Created] (IOTDB-290) Bug about threadlocal field in TSServiceImpl.java

[Yuan Tian (Jira)]

Yuan Tian created IOTDB-290:
---

 Summary: Bug about threadlocal field in TSServiceImpl.java
 Key: IOTDB-290
 URL: https://issues.apache.org/jira/browse/IOTDB-290
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: Yuan Tian


There are two fields named queryStatus and queryRet:
{code:java}
private ThreadLocal> queryStatus = new 
ThreadLocal<>();
private ThreadLocal> queryRet = new 
ThreadLocal<>();
{code}

They are used to save the PhysicalPlan and QueryDataSet of one query. However, 
the keys of both are sql string to be executed, so if the client execute the 
same sql in one connection, no matter whether the first execution has finished, 
the second sql execution will clear the first one's QueryDataSet.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

support measurements with same name

[Yuyuan KANG]

Hi, all


I'm now working on issue [IOTDB-243], to support 
measurements with same name and different types 
in the same storage group.


Best,
-
Yuyuan KANG

Re: Losing privileges on Travis

[Jialin Qiao]

Hi Tian Jiang,

Thanks, changing a browser works for me! it really bothers me a few days. 

--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "江天" 
> 发送时间: 2019-11-05 09:19:27 (星期二)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: Re: Losing privileges on Travis
> 
> Hi, 
> 
> It turns out that this if you change a browser (from Chrome to Safari in my 
> case), things will go back to normal. I don’t know who to blame, the browser 
> or Travis, but you can try using another browser if the same problem occurs.
> 
> Best,
> Tian Jiang
> 
> > 在 2019年10月22日，下午12:50，Tian Jiang  写道：
> > 
> > Hi,
> > 
> > As you can see from the picture, there was no restart button at all. 
> > However, that was yesterday and I just found I can restart now. I am not 
> > sure it was a bug of travis or anything, I just hope it won't happen again.
> > 
> > Best,
> > Tian Jiang
> > 
> > On 10/22/2019 12:28，Xiangdong Huang 
> >  wrote： 
> > Hi Tian Jiang,
> > 
> > I do not think so. It seems that Travis changed its UI.
> > 
> > So, I also can not restart a single job in the job list view of a build.
> > But if you click a job to see its details, you can find a button to restart
> > the job.
> > 
> > Best,
> > ---
> > Xiangdong Huang
> > School of Software, Tsinghua University
> > 
> > 黄向东
> > 清华大学 软件学院
> > 
> > 
> > Tian Jiang  于2019年10月21日周一 下午2:25写道：
> > 
> > Hi,
> > 
> > 
> > Well, things are getting a little better. I can restart a build now, but I
> > can't still restart a single job.
> > 
> > 
> > Tian Jiang
> > 
> > 
> > On 10/21/2019 13:20，Xiangdong Huang wrote：
> > Hi,
> > 
> > Ah? I think still have the privilege..
> > 
> > Best,
> > ---
> > Xiangdong Huang
> > School of Software, Tsinghua University
> > 
> > 黄向东
> > 清华大学 软件学院
> > 
> > 
> > 江天  于2019年10月21日周一 上午5:54写道：
> > 
> > Hi,
> > 
> > I just found I have lost some privileges on Travis, that is, I cannot
> > restart any builds or jobs.
> > 
> > Does anyone encounter the same problem or have the solution?
> > 
> > Thanks,
> > 
> > Tian Jiang
> > 
> > 
> > 
> 

[jira] [Closed] (IOTDB-273) Parallel Recovery

[Jialin Qiao (Jira)]

 [ 
https://issues.apache.org/jira/browse/IOTDB-273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jialin Qiao closed IOTDB-273.
-
Fix Version/s: 0.9.0
   Resolution: Fixed

> Parallel Recovery
> -
>
> Key: IOTDB-273
> URL: https://issues.apache.org/jira/browse/IOTDB-273
> Project: Apache IoTDB
>  Issue Type: Improvement
>Reporter: Jialin Qiao
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.9.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When restarting IoTDB, we recover each storage group one by one, which costs 
> 20min for 8GB WAL. 
> The restart can speed up by parallelly recovery storage groups.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Losing privileges on Travis

[江天]

Hi, 

It turns out that this if you change a browser (from Chrome to Safari in my 
case), things will go back to normal. I don’t know who to blame, the browser or 
Travis, but you can try using another browser if the same problem occurs.

Best,
Tian Jiang

> 在 2019年10月22日，下午12:50，Tian Jiang  写道：
> 
> Hi,
> 
> As you can see from the picture, there was no restart button at all. However, 
> that was yesterday and I just found I can restart now. I am not sure it was a 
> bug of travis or anything, I just hope it won't happen again.
> 
> Best,
> Tian Jiang
> 
> On 10/22/2019 12:28，Xiangdong Huang 
>  wrote： 
> Hi Tian Jiang,
> 
> I do not think so. It seems that Travis changed its UI.
> 
> So, I also can not restart a single job in the job list view of a build.
> But if you click a job to see its details, you can find a button to restart
> the job.
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
> 黄向东
> 清华大学 软件学院
> 
> 
> Tian Jiang  于2019年10月21日周一 下午2:25写道：
> 
> Hi,
> 
> 
> Well, things are getting a little better. I can restart a build now, but I
> can't still restart a single job.
> 
> 
> Tian Jiang
> 
> 
> On 10/21/2019 13:20，Xiangdong Huang wrote：
> Hi,
> 
> Ah? I think still have the privilege..
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
> 黄向东
> 清华大学 软件学院
> 
> 
> 江天  于2019年10月21日周一 上午5:54写道：
> 
> Hi,
> 
> I just found I have lost some privileges on Travis, that is, I cannot
> restart any builds or jobs.
> 
> Does anyone encounter the same problem or have the solution?
> 
> Thanks,
> 
> Tian Jiang
> 
> 
> 

Re: Working on refactoring metadata package

[Tian Jiang]

I think using long IDs in the distributed mode itself makes things much 
complicated in the first place.


Tian Jiang


On 11/4/2019 19:16，Xiangdong Huang wrote：
Hi,

3. Do we consider add a long id to each path? (is that helpful?)
As far as I know, long id isn't helpful... maybe @Yanzhe An could give us
more detailed information.
3. Using string Ids does not induce too much overhead as far as I am
concerned, not to mention the cost of maintaining the bidirectional index.

And, considering about the cluster mode, we need to use Consistent Hashing
to partition data and lookup data, right? If so, will an ID be better than
a string?

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

黄向东
清华大学 软件学院


Tian Jiang  于2019年11月4日周一 下午7:00写道：

1. Adding some entities may help us manage the metadata, while I don't
think mere renaming really helps much.


2. I don't think PTree helps at all. I would suggest remove useless codes
as much as possible, unless real situations where it is needed are found.
Even so, a careful redesign is probably necessary and current
implementation does not seem to survive.


3. Using string Ids does not induce too much overhead as far as I am
concerned, not to mention the cost of maintaining the bidirectional index.


On 11/4/2019 18:42，Jialin Qiao wrote：
Hi,

1. How about using sg, device, measurement directly in this module? (Or we
can provide interface named like these).

About the name of "storage group", maybe SeriesGroup or database is better.

2. How do you think about the legacy attributes in the module called pTree
(property tree, which is actually like the tag in InfluxDB), remove it? or
introduce reverse index?

pTree is a good properties, I suggest to reserve it. However, how to use
pTree and mTree simultaneously should be well defined by a PM :-)

3. Do we consider add a long id to each path? (is that helpful?)

As far as I know, long id isn't helpful... maybe @Yanzhe An could give us
more detailed information.

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

-原始邮件-
发件人: "Xiangdong Huang" 
发送时间: 2019-11-04 15:45:45 (星期一)
收件人: dev@iotdb.apache.org
抄送:
主题: Re: Working on refactoring metadata package

Hi,

Well I planed to do that actually, but I find I can not guarantee my
developing time.. So it is good to see that you want to do that.

Yes we need to discuss about how to refactor. At least there are something
to do:

1. How about using sg, device, measurement directly in this module? (Or we
can provide interface named like these).
2. How do you think about the legacy attributes in the module called pTree
(property tree, which is actually like the tag in InfluxDB), remove it? or
introduce reverse index?
3. Do we consider add a long id to each path? (is that helpful?)

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

黄向东
清华大学 软件学院


Tian Jiang  于2019年11月4日周一 上午11:54写道：

Greetings,


As you may know, the metadata package is the last package that is not
refactored compared with other packages. These codes are old and some are
poorly organized and may be inefficient. Nevertheless, new bloods are
coming in and such codes are uneasy for them to understand. So I plan to
refactor the metadata package in order to remove the unused or inefficient
codes.


I don't have a hard standard or rule. I will just see what I can do.  And
I will check out a branch "refactor_metadata" for this. You are welcomed to
join this job and discuss in this thread what should be refactored and how.


Best,
Tian Jiang

Re: Working on refactoring metadata package

[Xiangdong Huang]

Hi,

> 3. Do we consider add a long id to each path? (is that helpful?)
> As far as I know, long id isn't helpful... maybe @Yanzhe An could give us
more detailed information.
> 3. Using string Ids does not induce too much overhead as far as I am
concerned, not to mention the cost of maintaining the bidirectional index.

And, considering about the cluster mode, we need to use Consistent Hashing
to partition data and lookup data, right? If so, will an ID be better than
a string?

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Tian Jiang  于2019年11月4日周一 下午7:00写道：

> 1. Adding some entities may help us manage the metadata, while I don't
> think mere renaming really helps much.
>
>
> 2. I don't think PTree helps at all. I would suggest remove useless codes
> as much as possible, unless real situations where it is needed are found.
> Even so, a careful redesign is probably necessary and current
> implementation does not seem to survive.
>
>
> 3. Using string Ids does not induce too much overhead as far as I am
> concerned, not to mention the cost of maintaining the bidirectional index.
>
>
> On 11/4/2019 18:42，Jialin Qiao wrote：
> Hi,
>
> 1. How about using sg, device, measurement directly in this module? (Or we
> can provide interface named like these).
>
> About the name of "storage group", maybe SeriesGroup or database is better.
>
> 2. How do you think about the legacy attributes in the module called pTree
> (property tree, which is actually like the tag in InfluxDB), remove it? or
> introduce reverse index?
>
> pTree is a good properties, I suggest to reserve it. However, how to use
> pTree and mTree simultaneously should be well defined by a PM :-)
>
> 3. Do we consider add a long id to each path? (is that helpful?)
>
> As far as I know, long id isn't helpful... maybe @Yanzhe An could give us
> more detailed information.
>
> Best,
> --
> Jialin Qiao
> School of Software, Tsinghua University
>
> 乔嘉林
> 清华大学 软件学院
>
> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-11-04 15:45:45 (星期一)
> 收件人: dev@iotdb.apache.org
> 抄送:
> 主题: Re: Working on refactoring metadata package
>
> Hi,
>
> Well I planed to do that actually, but I find I can not guarantee my
> developing time.. So it is good to see that you want to do that.
>
> Yes we need to discuss about how to refactor. At least there are something
> to do:
>
> 1. How about using sg, device, measurement directly in this module? (Or we
> can provide interface named like these).
> 2. How do you think about the legacy attributes in the module called pTree
> (property tree, which is actually like the tag in InfluxDB), remove it? or
> introduce reverse index?
> 3. Do we consider add a long id to each path? (is that helpful?)
>
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
>
> 黄向东
> 清华大学 软件学院
>
>
> Tian Jiang  于2019年11月4日周一 上午11:54写道：
>
> Greetings,
>
>
> As you may know, the metadata package is the last package that is not
> refactored compared with other packages. These codes are old and some are
> poorly organized and may be inefficient. Nevertheless, new bloods are
> coming in and such codes are uneasy for them to understand. So I plan to
> refactor the metadata package in order to remove the unused or inefficient
> codes.
>
>
> I don't have a hard standard or rule. I will just see what I can do.  And
> I will check out a branch "refactor_metadata" for this. You are welcomed to
> join this job and discuss in this thread what should be refactored and how.
>
>
> Best,
> Tian Jiang
>

[jira] [Closed] (IOTDB-237) client fetches all data in spite of maxPrintRowCount

[Lei Rui (Jira)]

 [ 
https://issues.apache.org/jira/browse/IOTDB-237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lei Rui closed IOTDB-237.
-
Resolution: Duplicate

> client fetches all data in spite of maxPrintRowCount
> 
>
> Key: IOTDB-237
> URL: https://issues.apache.org/jira/browse/IOTDB-237
> Project: Apache IoTDB
>  Issue Type: Improvement
>Reporter: Lei Rui
>Priority: Major
>
> In line 202-208 of AbstractClient.java, the client fetches all data from the 
> server, showing maxPrintRowCount rows.
> In line 223, the client shows the total row count of the whole queried result.
> Can the client only fetch data within the maxPrintRowCount constraint and 
> still be able to know the total result count?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Working on refactoring metadata package

[Tian Jiang]

1. Adding some entities may help us manage the metadata, while I don't think 
mere renaming really helps much.


2. I don't think PTree helps at all. I would suggest remove useless codes as 
much as possible, unless real situations where it is needed are found. Even so, 
a careful redesign is probably necessary and current implementation does not 
seem to survive.


3. Using string Ids does not induce too much overhead as far as I am concerned, 
not to mention the cost of maintaining the bidirectional index. 


On 11/4/2019 18:42，Jialin Qiao wrote：
Hi,

1. How about using sg, device, measurement directly in this module? (Or we
can provide interface named like these).

About the name of "storage group", maybe SeriesGroup or database is better.

2. How do you think about the legacy attributes in the module called pTree
(property tree, which is actually like the tag in InfluxDB), remove it? or
introduce reverse index?

pTree is a good properties, I suggest to reserve it. However, how to use pTree 
and mTree simultaneously should be well defined by a PM :-)

3. Do we consider add a long id to each path? (is that helpful?)

As far as I know, long id isn't helpful... maybe @Yanzhe An could give us more 
detailed information.

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

-原始邮件-
发件人: "Xiangdong Huang" 
发送时间: 2019-11-04 15:45:45 (星期一)
收件人: dev@iotdb.apache.org
抄送:
主题: Re: Working on refactoring metadata package

Hi,

Well I planed to do that actually, but I find I can not guarantee my
developing time.. So it is good to see that you want to do that.

Yes we need to discuss about how to refactor. At least there are something
to do:

1. How about using sg, device, measurement directly in this module? (Or we
can provide interface named like these).
2. How do you think about the legacy attributes in the module called pTree
(property tree, which is actually like the tag in InfluxDB), remove it? or
introduce reverse index?
3. Do we consider add a long id to each path? (is that helpful?)

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

黄向东
清华大学 软件学院


Tian Jiang  于2019年11月4日周一 上午11:54写道：

Greetings,


As you may know, the metadata package is the last package that is not
refactored compared with other packages. These codes are old and some are
poorly organized and may be inefficient. Nevertheless, new bloods are
coming in and such codes are uneasy for them to understand. So I plan to
refactor the metadata package in order to remove the unused or inefficient
codes.


I don't have a hard standard or rule. I will just see what I can do.  And
I will check out a branch "refactor_metadata" for this. You are welcomed to
join this job and discuss in this thread what should be refactored and how.


Best,
Tian Jiang

Re: Working on refactoring metadata package

[Jialin Qiao]

Hi,

> 1. How about using sg, device, measurement directly in this module? (Or we
> can provide interface named like these).

About the name of "storage group", maybe SeriesGroup or database is better.

> 2. How do you think about the legacy attributes in the module called pTree
> (property tree, which is actually like the tag in InfluxDB), remove it? or
> introduce reverse index?

pTree is a good properties, I suggest to reserve it. However, how to use pTree 
and mTree simultaneously should be well defined by a PM :-)

> 3. Do we consider add a long id to each path? (is that helpful?)

As far as I know, long id isn't helpful... maybe @Yanzhe An could give us more 
detailed information.

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Xiangdong Huang" 
> 发送时间: 2019-11-04 15:45:45 (星期一)
> 收件人: dev@iotdb.apache.org
> 抄送: 
> 主题: Re: Working on refactoring metadata package
> 
> Hi,
> 
> Well I planed to do that actually, but I find I can not guarantee my
> developing time.. So it is good to see that you want to do that.
> 
> Yes we need to discuss about how to refactor. At least there are something
> to do:
> 
> 1. How about using sg, device, measurement directly in this module? (Or we
> can provide interface named like these).
> 2. How do you think about the legacy attributes in the module called pTree
> (property tree, which is actually like the tag in InfluxDB), remove it? or
> introduce reverse index?
> 3. Do we consider add a long id to each path? (is that helpful?)
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
>  黄向东
> 清华大学 软件学院
> 
> 
> Tian Jiang  于2019年11月4日周一 上午11:54写道：
> 
> > Greetings,
> >
> >
> > As you may know, the metadata package is the last package that is not
> > refactored compared with other packages. These codes are old and some are
> > poorly organized and may be inefficient. Nevertheless, new bloods are
> > coming in and such codes are uneasy for them to understand. So I plan to
> > refactor the metadata package in order to remove the unused or inefficient
> > codes.
> >
> >
> > I don't have a hard standard or rule. I will just see what I can do.  And
> > I will check out a branch "refactor_metadata" for this. You are welcomed to
> > join this job and discuss in this thread what should be refactored and how.
> >
> >
> > Best,
> > Tian Jiang

Re: Using exception instead of returning true/false

[Jialin Qiao]

Hi,

In the QueryProcessExecutor class, many non-query methods return a boolean. 
However, we do not use them. Using exception will make the codes clear and easy 
to maintain.

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "南京大学软件学院薛恺丰" <827011...@qq.com>
> 发送时间: 2019-11-04 15:43:08 (星期一)
> 收件人: dev 
> 抄送: 
> 主题: Using exception instead of returning true/false
> 
> Greeting!I'm working on the issue IOTDB-218, welcome to discuss with me.

Re: Is there anything needed to be developed now?

[Jialin Qiao]

Hi,

I can't assign this issue... leaving a comment under the issue is enough :-)

Best,
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Jack Tsai" 
> 发送时间: 2019-11-04 07:41:42 (星期一)
> 收件人: "dev@iotdb.apache.org" 
> 抄送: 
> 主题: 回覆: Is there anything needed to be developed now?
> 
> Hi Jialin,
> 
> I think I could take this issue. Please assign it to me on JIRA. I have 
> already commented there.
> 
> Best regards,
> Jack Tsai
> 
> 
> 寄件者: Jialin Qiao 
> 寄件日期: Saturday, November 2, 2019 3:16:15 AM
> 收件者: dev@iotdb.apache.org 
> 主旨: Re: Is there anything needed to be developed now?
> 
> Hi Jack,
> 
> Maybe this issue is suitable for you:
> 
> https://issues.apache.org/jira/browse/IOTDB-279
> 
> Best,
> --
> Jialin Qiao
> School of Software, Tsinghua University
> 
> 乔嘉林
> 清华大学 软件学院
> 
> > -原始邮件-
> > 发件人: "Jack Tsai" 
> > 发送时间: 2019-11-01 09:01:45 (星期五)
> > 收件人: "dev@iotdb.apache.org" 
> > 抄送:
> > 主题: Is there anything needed to be developed now?
> >
> > Hi,
> >
> > I have finished the function about querying timeseries and devices 
> > recently. It is glad to see that IoTDB keeps releasing new releases these 
> > days.
> >
> > Is there anything I could help now?  Maybe implementing some functions that 
> > is needed for the next release.
> >
> > Best regards,
> > Jack Tsai