[jira] [Commented] (IOTDB-606) [Distributed] Replace raw logs in MetaSnapshot
[ https://issues.apache.org/jira/browse/IOTDB-606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091145#comment-17091145 ] Tian Jiang commented on IOTDB-606: -- I think the idea is overall clear enough. Looking forward to your implementation and evaluation. > [Distributed] Replace raw logs in MetaSnapshot > -- > > Key: IOTDB-606 > URL: https://issues.apache.org/jira/browse/IOTDB-606 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Tian Jiang >Priority: Major > Labels: cluster, metadata, snapshot > > The current MetaSnapshot is using the simplest way, storing the raw committed > logs. It would be more efficient to replace the logs with compact structures > like the partition table and other objects that will be affected by meta logs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IOTDB-616) Grammar mistake in IOTDB official website
[ https://issues.apache.org/jira/browse/IOTDB-616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091094#comment-17091094 ] Xiangdong Huang commented on IOTDB-616: --- Hi [~sulthan309], Thanks for reminding us. I fixed it now. If you find other errors, you can try to fix it as your first (and more) contribution(s) :D If you want to do that yourself, just locate where the content on the website is, modify it, and commit a PR to us. The content on the website is located in: - site/src/main/.vuepress/theme/global-components ( for example, the home page content is on site/src/main/.vuepress/theme/global-components/IoTDB.vue) - docs/UserGuide/*.md - docs/SystemDesign/*.md Look forward to your more contribution! > Grammar mistake in IOTDB official website > - > > Key: IOTDB-616 > URL: https://issues.apache.org/jira/browse/IOTDB-616 > Project: Apache IoTDB > Issue Type: Bug > Components: Website >Reporter: Mohammed Sulthan >Priority: Minor > Attachments: 2020-04-23_21h21_45.png > > > In the official website of IOTDB ([https://iotdb.apache.org/]) I found the > following error. > * In homepage, in 'Scenarios' carousal, first slide contains "if supports > Java or Go (in developing)". It should be "if supports Java or Go (in > development)". > I have screenshot for reference. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IOTDB-606) [Distributed] Replace raw logs in MetaSnapshot
[ https://issues.apache.org/jira/browse/IOTDB-606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091091#comment-17091091 ] Houliang Qi commented on IOTDB-606: --- The operations that can cause the contents of the partition table to change in the system are as follows: 1. Add a node; 2. Remove a node; The main reasons when a node needs to pull metasnapshot are as follows; 1. New nodes are added: 2. Restart after downtime, the meta information of this node is far away from the leader. 3. The new network partition node rejoins the cluster, the meta information of this node has been far away from the leader. For 1, no request will come before the new partition table is applied. So just apply the partition table directly. For 2 and 3, if the request is being routed to this node, because the partition table information is old, the metadata obtained by the metamember or datamember is also wrong. In this case, the operation will definitely fail, so let the upper layer retry. This node can directly replace the partition table. Before the replacement of the partition table is completed, all operations are blocked(emptying flow). The above is to consider the case of adding only one node or deleting one node at a time. Let us consider the case of adding or deleting multiple nodes. Since all operations are performed sequentially at the leader node, the leader has the newest partition table, Raft guarantees that the partition table given by the leader to the follower must be accurate. So in this case, for follower, it is the same as the addition and deletion of a node. Please leave your opinion, thanks. > [Distributed] Replace raw logs in MetaSnapshot > -- > > Key: IOTDB-606 > URL: https://issues.apache.org/jira/browse/IOTDB-606 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Tian Jiang >Priority: Major > Labels: cluster, metadata, snapshot > > The current MetaSnapshot is using the simplest way, storing the raw committed > logs. It would be more efficient to replace the logs with compact structures > like the partition table and other objects that will be affected by meta logs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IOTDB-560) Add Flink Connector Support for TsFile
[ https://issues.apache.org/jira/browse/IOTDB-560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091074#comment-17091074 ] sunjincheng commented on IOTDB-560: --- Add the link of second PR #1084. > Add Flink Connector Support for TsFile > -- > > Key: IOTDB-560 > URL: https://issues.apache.org/jira/browse/IOTDB-560 > Project: Apache IoTDB > Issue Type: New Feature > Components: Others >Reporter: sunjincheng >Assignee: Wei Zhong >Priority: Major > Labels: pull-request-available > Fix For: 0.9.2 > > Time Spent: 20m > Remaining Estimate: 0h > > TsFile is a columnar storage file format in Apache IoTDB. It is designed for > time series data and supports efficient compression and query and is easy to > be integrated into big data processing frameworks. > Apache Flink is a framework and distributed processing engine for stateful > computations over unbounded and bounded data streams and becoming more and > more popular in IOT scenes. So, it would be great to integrate IoTDB and > Flink. > In this JIRA I would like to introduce the TsFile Flink Connector, i.e., > allows Flink to read, write TsFile. > More detail can be found in discussion thread [1], or in the IoTDB wiki [2] > [1]https://lists.apache.org/thread.html/r6dd6afe4e8e4ca42e3ddfbc80609597788f90b214e7a81788c3b51b3%40%3Cdev.iotdb.apache.org%3E > [2]https://cwiki.apache.org/confluence/display/IOTDB/%5BImprovement+Proposal%5D+Add+Flink+Connector+Support+for+TsFile -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-616) Grammar mistake in IOTDB official website
Mohammed Sulthan created IOTDB-616: -- Summary: Grammar mistake in IOTDB official website Key: IOTDB-616 URL: https://issues.apache.org/jira/browse/IOTDB-616 Project: Apache IoTDB Issue Type: Bug Components: Website Reporter: Mohammed Sulthan Attachments: 2020-04-23_21h21_45.png In the official website of IOTDB ([https://iotdb.apache.org/]) I found the following error. * In homepage, in 'Scenarios' carousal, first slide contains "if supports Java or Go (in developing)". It should be "if supports Java or Go (in development)". I have screenshot for reference. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IOTDB-348) [Distributed] Support more non-query operations (log types)
[ https://issues.apache.org/jira/browse/IOTDB-348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17090524#comment-17090524 ] Jialin Qiao commented on IOTDB-348: --- update(add limit in show timeseries) CRUD: * -DeletePlan- ** delete from root.ln.wf02.wt02.status where time<=2017-11-01T16:26:00; ** delete from root.ln.wf02.wt02.status, root.sgcc.wf03.status where time<=2017-11-01T16:26:00; * UpdatePlan (does not support) System: * -AuthorPlan (K)- ** CREATE USER ln_write_user 'write_pwd'; ** CREATE USER sgcc_write_user 'write_pwd'; ** LIST USER; ** GRANT USER ln_write_user PRIVILEGES 'INSERT_TIMESERIES' on root.ln ** GRANT USER sgcc_write_user PRIVILEGES 'INSERT_TIMESERIES' on root.sgcc * -CountPlan (K)- ** count timeseries root.* ** count timeseries root group by LEVEL=1 ** count nodes root level=2 * -DataAuthPlan- * ** grant watermark_embedding to Alice ** revoke watermark_embedding from Alice * -DeleteStorageGroupPlan (K)- ** DELETE STORAGE GROUP root.h1 * -DeleteTimeSeriesPlan (K)- ** DELETE TIMESERIES root.h1.w1.k1.s1; ** DELETE TIMESERIES root.h1.h2.* ** DELETE TIMESERIES root.h1.h2.h3, root.h1.h2.h4 * LoadConfigurationPlan (K) * LoadDataPlan (K) * OperateFilePlan * -SetTTLPlan- ** set ttl to root.ln ** unset ttl to root.ln * -SHowChildPathsPlan (W)- * ShowPlan (W) * -ShowTimeSeriesPlan (K)- ** show timeseries root ** show timeseries root where tag=value ** show timeseries root where tag contains value ** show timeseries root limit 10 offset 10 * -ShowTTLPlan (W)- ** show ttl on root.ln * -ShowStorageGroup (W)- ** show storage group > [Distributed] Support more non-query operations (log types) > --- > > Key: IOTDB-348 > URL: https://issues.apache.org/jira/browse/IOTDB-348 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Tian Jiang >Priority: Major > Labels: distributed, pull-request-available > Attachments: image-2020-01-30-21-18-42-831.png > > Time Spent: 50m > Remaining Estimate: 0h > > Currently supported operations: > create storage group > create timeseries > single row insertion > Please link to and reply to this issue if you added any new functionalities. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IOTDB-606) [Distributed] Replace raw logs in MetaSnapshot
[ https://issues.apache.org/jira/browse/IOTDB-606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17090434#comment-17090434 ] Tian Jiang commented on IOTDB-606: -- Great job. The first four are basically fine and applying them should be almost no problem. However, the applying of a new partition table should be careful enough because it also involves the changes in membership and the distribution of slots. Just replacing the old table with the new one is not enough. I hope that you can describe the procedure carefully and discuss potential problems that may arise during it. > [Distributed] Replace raw logs in MetaSnapshot > -- > > Key: IOTDB-606 > URL: https://issues.apache.org/jira/browse/IOTDB-606 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Tian Jiang >Priority: Major > Labels: cluster, metadata, snapshot > > The current MetaSnapshot is using the simplest way, storing the raw committed > logs. It would be more efficient to replace the logs with compact structures > like the partition table and other objects that will be affected by meta logs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
