Re: [VOTE] Apache IoTDB 1.3.1 RC2 release
Hi, +1 (binding) The source release: apache headers [ok] signatures and hashes [ok] LICENSE and NOTICE [ok] no jar files [ok] could compile from source: ./mvnw.sh clean install [ok] The binary distribution: version number in CLI [ok] signatures and hashes [ok] start in mac, jdk8 [ok] statements executed successfully: [ok] SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); insert into root.turbine.d2(time,s0) values(now(),1); insert into root.turbine.d2(time,s0) values(now(),2); insert into root.turbine.d2(time,s0) values(now(),3); select ** from root; select ** from root align by device; Jialin Qiao Yuan Tian 于2024年4月22日周一 08:52写道： > > Hi, > > +1 (binding) > > The source release: > apache headers [ok] > signatures and hashes [ok] > LICENSE and NOTICE [ok] > no jar files [ok] > could compile from source: ./mvnw.sh clean install [ok] > > The binary distribution: > version number in CLI [ok] > signatures and hashes [ok] > start in mac, jdk8 [ok] > statements executed successfully: [ok] > > SET STORAGE GROUP TO root.turbine; > CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; > insert into root.turbine.d1(timestamp,s0) values(1,1); > insert into root.turbine.d1(timestamp,s0) values(2,2); > insert into root.turbine.d1(timestamp,s0) values(3,3); > insert into root.turbine.d2(time,s0) values(now(),1); > insert into root.turbine.d2(time,s0) values(now(),2); > insert into root.turbine.d2(time,s0) values(now(),3); > select ** from root; > select ** from root align by device; > > > Best, > - > Yuan Tian > > On Fri, Apr 19, 2024 at 5:14 PM Haonan Hou wrote: > > > Hi all, > > > > Apache IoTDB 1.3.1 has been staged under [2] and it’s time to vote > > on accepting it for release. All Maven artifacts are available under [1]. > > Voting will be open for 72hr. > > A minimum of 3 binding +1 votes and more binding +1 than binding -1 > > are required to pass. > > > > Release tag: v1.3.1 > > Hash for the release tag: 214695df59915884ca43767f944ed7b921763864 > > > > Before voting +1, PMC members are required to download > > the signed source code package, compile it as provided, and test > > the resulting executable on their own platform, along with also > > verifying that the package meets the requirements of the ASF policy > > on releases. [3] > > > > You can achieve the above by following [4]. > > > > [ ] +1 accept (indicate what you validated - e.g. performed the > > non-RM items in [4]) > > [ ] -1 reject (explanation required) > > > > > > [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1157 > > [2] https://dist.apache.org/repos/dist/dev/iotdb/1.3.1/rc2 > > [3] https://www.apache.org/dev/release.html#approving-a-release > > [4] > > https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release > > [5] https://github.com/apache/iotdb/blob/v1.3.1/RELEASE_NOTES.md > > [6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS > > > > Best, > > > > Haonan Hou > >
Re: Remove TsFile module in IoTDB Project
Looks good! Jialin Qiao Yuan Tian 于2024年4月19日周五 22:01写道： > > Hi all, > > I've just merged this pr(https://github.com/apache/iotdb/pull/12283), now > tsfile module source code have all been removed from IoTDB project and > using pom to depend on TsFile Project. > > > > Best regards, > -- > Yuan Tian
Re: Archive apache/iotdb-web-workbench repo?
+1 for archive Jialin Qiao Haonan Hou 于2024年4月18日周四 11:23写道： > > Hi all, > > We received some security issues about apache/iotdb-web-workbench recently. > However, we don't have enough maintainers to fix them. The repo has no new > commit since Apr 3, 2023. There is an option that we can archive that repo. > > Any idea? > > Best, > Haonan Hou
Re: Splitting up the repos
Hi, The examples could be classified into two parts (1) Connector example: flink, hadoop, kafka, pulsar, rabiitmq, rocketmq (2) IoTDB native api example: jdbc, mqtt, pipe, rest, schema, session, trigger, udf For (1), we could move into extra repo. For (2) , they should be retained in the IoTDB repo. Jialin Qiao Christofer Dutz 于2024年4月15日周一 17:56写道： > > Hmpf, > > a little bit sooner reply would have been good … I’m already done with the > changes, also with moving all examples and the parts of the distribution > bundling the connectors. > > I do think also moving the examples is a good idea. Usually, examples pull in > all sorts of dependencies, which show up on vulnerability reports. Also do we > have some examples that refer to stuff we now moved out of the main repo, > we’d be getting a cyclic dependency from that, so we would have to split up > the examples in that case. > > So, if we were to vote on this (which we can) I would vote +1 on moving all > examples out of the main repo. > > Chris > > > Von: Jialin Qiao > Datum: Montag, 15. April 2024 um 11:20 > An: dev@iotdb.apache.org > Betreff: Re: Splitting up the repos > Hi, > > 1. Which Parts: Only iotdb-connector need to be moved, distribution > and examples will impact the release and users. > 2. How to split up: I prefer【Simply ignore the history, copy the > files to the new repo and delete them from the old】. > > Jialin Qiao > > Christofer Dutz 于2024年4月15日周一 16:27写道： > > > > Hi all, > > > > > > So, I’ve set a tag on the main repository “before-moving-extras” > > (https://github.com/apache/iotdb/releases/tag/before-moving-extras) > > > > Also have I copied the content of the examples and integration modules to > > the new repo, duplicated the build there and updated the versions to > > artifacts in the main repo to reference a variable. > > > > The build in the extras seems to work, now I’ll have to strip out > > configurations, dependency management etc. for stuff that’s not needed in > > the extras and do the same in the main repo. > > > > > > > > Chris > > > > > > > > Von: Christofer Dutz > > Datum: Montag, 15. April 2024 um 09:22 > > An: dev@iotdb.apache.org > > Betreff: AW: Splitting up the repos > > Ok … > > > > So, no comment I will simply treat as lazy consensus, therefore I will move > > forward with tagging the main repo with the latest changes as last revision > > before the split and reference that in the commit to the new repo. > > Then I’ll simply copy over the files and delete them from the main repo. > > > > As with other projects however, I really dislike this form of workting > > together. Defaulting back to lazy consensus costs a lot of valuable time as > > I have to wait a reasonable amount of time. If I had gotten any “sure … I’m > > fine with you doing X” I could have long finished this. > > > > In the future it would be a lot better, if some people would actually reply. > > > > > > Chris > > > > > > > > Von: Christofer Dutz > > Datum: Donnerstag, 11. April 2024 um 10:36 > > An: dev@iotdb.apache.org > > Betreff: Splitting up the repos > > Hi all, > > > > so now that the new repo is created > > (https://gitbox.apache.org/repos/asf/iotdb-extras.git, but please don’t > > push anything there just yet), we would need to decide on which parts > > should be moved there. > > > > > > * “distribution”: Here I think we need to split the distribution. > > Keeping the distributions containing only core in the main repo and adding > > a new distribution module in the extras repo, that contains the downstream > > components. > > * “example” (which I would propose to rename to examples as it contains > > multiple) > > * “iotdb-connector” > > > > > > As it seems that in the integration-tests there are no tests testing the > > connectors, I guess we can leave that as it is. > > > > Now the problem is: There are multiple options to split up the repo and > > keeping the entire history. > > > > 1. Split out one directory in a separate branch and then merge all > > branches into an empty new one > > 2. Use the filter plugin to strip out all commits that match a regexp > > 3. Simply ignore the history, copy the files to the new repo and delete > > them from the old. > > > > 3 is the simples, but the person doing the move will be marked as author. > > In general this is not that problematic, as the integration modules and the > > examples are usually not that complex, but I would understand, if people > > wanted to keep the history. > > > > Option 1 is probably the most work, but the most robust option, as with > > option 2, I had to give up when doing the PLC4X split as there were bugs > > and issues in the tooling. > > > > So, if nobody objects and we’ve decided on what should be moved, I > > personally would opt for option 3. > > > > Chris
Re: Splitting up the repos
Hi, 1. Which Parts: Only iotdb-connector need to be moved, distribution and examples will impact the release and users. 2. How to split up: I prefer【Simply ignore the history, copy the files to the new repo and delete them from the old】. Jialin Qiao Christofer Dutz 于2024年4月15日周一 16:27写道： > > Hi all, > > > So, I’ve set a tag on the main repository “before-moving-extras” > (https://github.com/apache/iotdb/releases/tag/before-moving-extras) > > Also have I copied the content of the examples and integration modules to the > new repo, duplicated the build there and updated the versions to artifacts in > the main repo to reference a variable. > > The build in the extras seems to work, now I’ll have to strip out > configurations, dependency management etc. for stuff that’s not needed in the > extras and do the same in the main repo. > > > > Chris > > > > Von: Christofer Dutz > Datum: Montag, 15. April 2024 um 09:22 > An: dev@iotdb.apache.org > Betreff: AW: Splitting up the repos > Ok … > > So, no comment I will simply treat as lazy consensus, therefore I will move > forward with tagging the main repo with the latest changes as last revision > before the split and reference that in the commit to the new repo. > Then I’ll simply copy over the files and delete them from the main repo. > > As with other projects however, I really dislike this form of workting > together. Defaulting back to lazy consensus costs a lot of valuable time as I > have to wait a reasonable amount of time. If I had gotten any “sure … I’m > fine with you doing X” I could have long finished this. > > In the future it would be a lot better, if some people would actually reply. > > > Chris > > > > Von: Christofer Dutz > Datum: Donnerstag, 11. April 2024 um 10:36 > An: dev@iotdb.apache.org > Betreff: Splitting up the repos > Hi all, > > so now that the new repo is created > (https://gitbox.apache.org/repos/asf/iotdb-extras.git, but please don’t push > anything there just yet), we would need to decide on which parts should be > moved there. > > > * “distribution”: Here I think we need to split the distribution. Keeping > the distributions containing only core in the main repo and adding a new > distribution module in the extras repo, that contains the downstream > components. > * “example” (which I would propose to rename to examples as it contains > multiple) > * “iotdb-connector” > > > As it seems that in the integration-tests there are no tests testing the > connectors, I guess we can leave that as it is. > > Now the problem is: There are multiple options to split up the repo and > keeping the entire history. > > 1. Split out one directory in a separate branch and then merge all > branches into an empty new one > 2. Use the filter plugin to strip out all commits that match a regexp > 3. Simply ignore the history, copy the files to the new repo and delete > them from the old. > > 3 is the simples, but the person doing the move will be marked as author. In > general this is not that problematic, as the integration modules and the > examples are usually not that complex, but I would understand, if people > wanted to keep the history. > > Option 1 is probably the most work, but the most robust option, as with > option 2, I had to give up when doing the PLC4X split as there were bugs and > issues in the tooling. > > So, if nobody objects and we’ve decided on what should be moved, I personally > would opt for option 3. > > Chris
Re: New Table Model for IoTDB
Great, this will extend the usage scenario of IoTDB. Jialin Qiao Yuan Tian 于2024年4月7日周日 19:39写道： > > Hi all, > > As introduced in the official documentation( > https://iotdb.apache.org/UserGuide/latest/Basic-Concept/Data-Model-and-Terminology.html), > the previous modeling method of IoTDB was a tree model, which formed a full > path of a sequence from the root node to the leaf node, and a device path > from the root node to the second-to-last layer node. The previous query > syntax of IoTDB was very similar to the standard relational SQL, centered > on the sequence, with the sequence prefix name in `from clause` and the > sequence suffix name in `select clause`. This query syntax is not very > friendly to users who are accustomed to relational SQL, and they cannot > apply their previous query experience to IoTDB. > > Therefore, we are designing a new schema model called table model for > IoTDB. We will provide data to users in a table view in the same way as > relational databases. Each kind of device belongs to a table, and users can > use standard SQL to query this table, which greatly reduces the learning > curve of IoTDB. > > The functional specs for table model can be found in > https://timechor.feishu.cn/docx/C2eodP84VoJ0kuxgbhlc1fejnsh，and our dev > branch is ty/TableModelGrammar。Syntax Definition file for Table model can > be found > in > iotdb-core/relational-grammar/src/main/antlr4/org/apache/iotdb/db/relational/grammar/sql/RelationalSql.g4 > > To support the table model, we also need to change the current tsfile > format, so we need to upgrade tsfile version from V3 to V4, the new file > format for tsfile V4 can be seen in > https://apache-iotdb.feishu.cn/docx/QNeVd7mpVoWaFxxobopcCsw6ne5 > > Our development is currently at a very early stage, and we would like to > invite you to discuss the functionality of the table model. Your feedback > is valuable to us and will help us shape the development of this feature. > > > Best regards, > -- > Yuan Tian
Re: Splitting up the main repo?
+1 for moving these into extra repos. Jialin Qiao Xiangdong Huang 于2024年4月7日周日 09:40写道： > > > We surely also pull in a lot of potentially bad dependencies. > > and this may be a chance to re-check our dependencies and remove unnecessary.. > > --- > Xiangdong Huang > > Christofer Dutz 于2024年4月5日周五 19:13写道： > > > > Hi all, > > > > I just wanted to bring up one idea that we decided in the PLC4X project and > > seed the idea, if this would also be worth discussing here. > > > > So, we were seeing that our build kept on having sub-ideal CVE ratings as > > we had dependencies for which CVEs were reported. > > However, PLC4X itself has a very limited number of dependencies. The > > problem was that we had several “integration” modules, that pulled in > > Kafka, Calcite, Nifi and some Eclipse projects. > > Also did a lot of our examples pull in various third party libraries, for > > which also vulnerabilities were reported. > > > > We are currently in the process of splitting up our main repository into a > > main and an extras repository. > > The main contains the core of the project. The extras contains the > > examples, additional tools and integration modules (The ones with the many, > > many dependencies) > > This way we can get a much better secutity standing for the main repo. > > > > Would this also be a good idea for IoTDB? I know with our dependencies to: > > > > * Flink > > * Grafana > > * Hadoop > > * Hive > > * Spark > > * Zeppelin (this one is really bad when it comes to CVEs) > > * Pulsar (only examples) > > * RabbitMQ (only examples) > > * RocketMQ (only examples) > > > > We surely also pull in a lot of potentially bad dependencies. If we moved > > this out the same way we would probably have a much better CVE ranking. > > This might become problematic in the future as in Europe and in the US > > CRE/PLD and other initiatives are taking form. > > > > Chris
Anything needs PR to TsFile
Hi, We plan to release the first version 1.0.0 of TsFile, is there anything that needs PR? Jialin Qiao
Re: PRs about tsfile module
Hi, TsFile is an independent project and need its own release schedule and version, so no submodule is better. Jialin Qiao Giorgio Zoppi 于2024年1月6日周六 03:41写道： > > Ok, > thanks i will join that list. > > Il giorno ven 5 gen 2024 alle ore 18:17 Christofer Dutz < > christofer.d...@c-ware.de> ha scritto: > > > Generally that will definitely be discussed on the d...@tsfile.apache.org > > list ;-) > > > > Chris > > > > Gesendet von Outlook für Android<https://aka.ms/AAb9ysg> > > > > From: Giorgio Zoppi > > Sent: Friday, January 5, 2024 5:18:55 PM > > To: dev@iotdb.apache.org > > Subject: Re: PRs about tsfile module > > > > Ok, i like it. I also suggest to include docusaurs for the format docs. > > https://docusaurus.io/, > > so we can use it. Also what is the plan for python/cpp/rust format > > implementation? > > BR, > > Giorgio > > > > Il giorno ven 5 gen 2024 alle ore 12:51 Yuan Tian > > ha scritto: > > > > > Hi all, > > > > > > Since we've already set up our tsfile repo, so if you want to make some > > > changes to common-api and tsfile submodule in IoTDB repo, you also need > > to > > > submit another pr for tsfile repo. > > > > > > > > > Best regards, > > > > > > Yuan Tian > > > > > > > > > -- > > Life is a chess game - Anonymous. > > > > > -- > Life is a chess game - Anonymous.
Re: PRs about tsfile module
+1 Jialin Qiao Yuan Tian 于2024年1月5日周五 20:51写道： > > Hi all, > > Since we've already set up our tsfile repo, so if you want to make some > changes to common-api and tsfile submodule in IoTDB repo, you also need to > submit another pr for tsfile repo. > > > Best regards, > > Yuan Tian
Re: Is there a reason we're not deploying "library-udf"?
Hi, No reason, we could enable it. Jialin Qiao Christofer Dutz 于2024年1月4日周四 20:55写道： > > Hi all, > > while setting up Apache TsFile, I noticed, that we’re excluding “library-udf” > from SNAPSHOT deployment on Jenkins. Is there a reason for this? > > (Always to leave a comment in the code on stuff like that) > > Chris >
Re: [ANNOUNCE] Apache IoTDB 1.3.0 released
Congratulations! — Jialin Qiao Apache IoTDB PMC Haonan Hou 于2024年1月2日周二 09:47写道： > > The Apache IoTDB team is pleased to announce the release of Apache IoTDB > 1.3.0. > > Apache IoTDB (Database for Internet of Things) is an IoT native database > with high performance for data management and analysis, deployable on the > edge and the cloud. > > Apache IoTDB 1.3.0 is the latest version of the v1 series. Feel free to > upgrade > from earlier version. > > ## Features & Improvements > > - Optimize the permission module and support timeseries permission control > - Optimize heap and off-heap memory configuration in startup script > - Computed-type view timeseries support LAST queries > - Add pipe-related monitoring metrics > - Pipe rename 'Extractor' to 'Source' and 'Connector' to 'Sink' > - [IOTDB-6138] Support Negative timestamp > - [IOTDB-6193] Reject Node startup when loading configuration file failed > - [IOTDB-6194] Rename target_config_node_list to seed_config_node > - [IOTDB-6200] Change schema template to device template > - [IOTDB-6207] Add Write Point Metrics for load function > - [IOTDB-6208] Node error detection through broken thrift pipe > - [IOTDB-6217] When the number of time series reaches the upper limit, the > prompt message should be changed to prioritize using device templates > - [IOTDB-6218] Rename storage_query_schema_consensus_free_memory_proportion > to datanode_memory_proportion > - [IOTDB-6219] Fix the display problem of explain that the print result is > not aligned > - [IOTDB-6220] Pipe: Add check logic to avoid self-transmission > - [IOTDB-6222] Optimize the performance of Python client > - [IOTDB-6230] Add HEAPDUMP configuration in datanode-env.sh > - [IOTDB-6231]SchemaCache supports precise eviction > - [IOTDB-6232] Adding SSL function to dn_rpc_port > > ## Bugs > > - [IOTDB-6160] while using ` in target path, select into will throw error > - [IOTDB-6167] DataNode can't register to cluster when fetch system > configuration throws NPE > - [IOTDB-6168] ConfigNode register retry logic does not worked > - [IOTDB-6171] NPE will be thrown while printing FI with debug on > - [IOTDB-6184] Merge Sort finishes one iterator too long > - [IOTDB-6191] Fix group by year not considering leap years > - [IOTDB-6226] Fix the problem of inaccurate GC monitor detection at the > beginning and adjusting the alert threshold > - [IOTDB-6239] Show regions display error create time > > The full release note is available at: > https://dlcdn.apache.org/iotdb/1.3.0/RELEASE_NOTES.md > > The release is available for download at: > http://iotdb.apache.org/Download > > Maven artifacts for JDBC driver, session SDK, TsFile SDK, Spark-connector, > Hadoop-connector, Hive-connector and Flink-connector can be found at: > https://search.maven.org/search?q=3Dg:org.apache.iotdb > > > Regards, > The Apache IoTDB team
Re: Celebrating the Release of Ratis 3.0.0 on Our Community's Contributions
Hi, Thank William for the contribution! — Jialin Qiao Apache IoTDB PMC Chao Wang 于2023年12月27日周三 11:39写道： > > Hi ,Thank you for your contribution. > I have a suggetion , When upgrading the ratis version, consider the > compatibility of the iotdb upgrade from an earlier version to a later version. > > On 2023/12/26 13:15:19 William Song wrote: > > Dear Community Members, > > > > I am excited to announce the release of Ratis version 3.0.0, marking the > > first major version update since IoTDB began utilizing Ratis for consensus > > services. This release encompasses a multitude of new features, > > enhancements, and fixes, all of which are the contributions of the > > dedicated efforts of our IoTDB community. Reflecting on the past year, it's > > hard to imagine the remarkable journey that unfolded after several key > > members (Mr. Huang, Mr. Qiao, and Xinyu Tan) made the pivotal decision to > > utilize Ratis for data replication. > > > > I wish to extend my gratitude to each one of you for your contributions, > > whether direct or indirect. Our collective success is a testament to the > > strength of our community spirit, resonating perfectly with the Apache > > Software Foundation's ethos: “Community Over Code.” > > > > In the upcoming period, we plan to integrate these features and > > improvements into the master branch. Each addition will undergo backtesting > > and validation to ensure optimal performance and reliability. Your feedback > > and insights are invaluable to us, and I eagerly look forward to your > > thoughts and suggestions. > > > > Thank you once again for your support and dedication. Together, we are > > shaping a brighter future for our project. > > > > > > > > Warm regards, > > William
Re: [VOTE] Apache IoTDB 1.3.0 RC1 release
Hi, +1 (binding) The source release: apache headers [ok] signatures and hashes [ok] LICENSE and NOTICE [ok] no jar files [ok] could compile from source: ./mvnw.sh clean install [ok] The binary distribution: version number in CLI [ok] signatures and hashes [ok] start in mac, jdk8 [ok] statements executed successfully: [ok] SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select ** from root; — Jialin Qiao Apache IoTDB PMC Haonan Hou 于2023年12月27日周三 12:33写道： > > Hi all, > > Apache IoTDB 1.3.0 has been staged under [2] and it’s time to vote > on accepting it for release. All Maven artifacts are available under [1]. > Voting will be open for 72hr. > A minimum of 3 binding +1 votes and more binding +1 than binding -1 > are required to pass. > > Release tag: v1.3.0 > Hash for the release tag: fe787cac2ddd874b44482a80306d9d6ae333143c > > Before voting +1, PMC members are required to download > the signed source code package, compile it as provided, and test > the resulting executable on their own platform, along with also > verifying that the package meets the requirements of the ASF policy > on releases. [3] > > You can achieve the above by following [4]. > > [ ] +1 accept (indicate what you validated - e.g. performed the > non-RM items in [4]) > [ ] -1 reject (explanation required) > > > [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1154 > [2] https://dist.apache.org/repos/dist/dev/iotdb/1.3.0/rc1 > [3] https://www.apache.org/dev/release.html#approving-a-release > [4] > https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release > [5] https://github.com/apache/iotdb/blob/v1.3.0/RELEASE_NOTES.md > [6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS > > Best, > > Haonan Hou
Re: Release schedule
Hi, If I understand correctly, this is just a change of package namespace. So users may need to upgrade their imports. We could do this in 1.3.1, which will be checkout at about 2023.12.30. Thanks, — Jialin Qiao Apache IoTDB PMC Christofer Dutz 于2023年11月16日周四 14:11写道： > > About the compatability. > > Most apache projects have finished the migration from Javax to Jakarta. I > think we should do this as soon as possible, as we are cutting us off from a > number of dependency updates. > > This change however is a breaking change. > > I was going to propose to do that for a first release of 1.3.0, but as that > is now officially already cut, when do we want to do that? > > Chris > > Gesendet von Outlook für Android<https://aka.ms/AAb9ysg> > > From: Jialin Qiao > Sent: Wednesday, November 15, 2023 7:17:40 AM > To: dev@iotdb.apache.org > Subject: Re: Release schedule > > Hi, sorry for a typo: > > Then checkout rc/1.3.1 from master at 2023-12-30, and release 1.3.1 > after 2024-1-30. > > Thanks, > Jialin Qiao > > On 2023/11/15 05:46:11 Jialin Qiao wrote: > > Hi, > > > > I come to share about our release schedule. > > > > Currently, we are all working on master branch, and checkout a rc > > branch from master for release. > > So it is essential to ensure that the master branch remains compatible > > with the previous version at all times. > > > > The release will be done every two months. > > > > As we checkout rc/1.3.0 from master at 2023-10-30, and will release > > 1.3.0 after 2023-11-30. > > Then checkout rc/1.3.1 from master at 2023-12-30, and release 1.3.0 > > after 2024-1-30. > > .. > > > > > > Thanks, > > — > > Jialin Qiao > > Apache IoTDB PMC > >
Re: Apache TsFile
Good news! Thank Chris! — Jialin Qiao Apache IoTDB PMC Christofer Dutz 于2023年11月16日周四 16:52写道： > > Hi all, > > so at yesterdays board meeting the Apache TsFile TLP proposal was accepted. I > am currently requesting things to be created. The mailinglists: dev, commits > and private will be the first things that need to be created and I initiated > that and need to wait for them being created. > > As soon as that’s done I’ll request the git repo for the project. > > I’ll keep you posted on when it makes sense to sign up to the mailinglists. > > > Chris
Re: Release schedule
Hi, sorry for a typo: Then checkout rc/1.3.1 from master at 2023-12-30, and release 1.3.1 after 2024-1-30. Thanks, Jialin Qiao On 2023/11/15 05:46:11 Jialin Qiao wrote: > Hi, > > I come to share about our release schedule. > > Currently, we are all working on master branch, and checkout a rc > branch from master for release. > So it is essential to ensure that the master branch remains compatible > with the previous version at all times. > > The release will be done every two months. > > As we checkout rc/1.3.0 from master at 2023-10-30, and will release > 1.3.0 after 2023-11-30. > Then checkout rc/1.3.1 from master at 2023-12-30, and release 1.3.0 > after 2024-1-30. > ...... > > > Thanks, > — > Jialin Qiao > Apache IoTDB PMC >
Release schedule
Hi, I come to share about our release schedule. Currently, we are all working on master branch, and checkout a rc branch from master for release. So it is essential to ensure that the master branch remains compatible with the previous version at all times. The release will be done every two months. As we checkout rc/1.3.0 from master at 2023-10-30, and will release 1.3.0 after 2023-11-30. Then checkout rc/1.3.1 from master at 2023-12-30, and release 1.3.0 after 2024-1-30. .. Thanks, — Jialin Qiao Apache IoTDB PMC
Re: Add -XX:+ExitOnOutOfMemoryError as default JVM parameter in datanode-env and confignode-env
+1 for ExitOnOutOfMemoryError. Dump could be an option, disabled by default. — Jialin Qiao Apache IoTDB PMC Yuan Tian 于2023年11月14日周二 14:10写道： > > Hi, > > +HeapDumpOnOutOfMemroyError is not provided as default jvm args, I've > comment that out, just to provide convenience for DBAs when they want to > use this parameter. > > > On Tue, Nov 14, 2023 at 2:00 PM ZhangJian He wrote: > > > +1 for ExitOnOutOfMemoryError. > > +HeapDumpOnOutOfMemroyError may produce very large files. How to clean up > > these old files? WDYT? > > > > Thanks > > ZhangJian He > > > > > > On Tue, 14 Nov 2023 at 12:04, Gaofei Cao wrote: > > > > > +1, > > > > > > `-XX:+ExitOnOutOfMemoryError` parameter can avoid the loss of some > > > key threads, it will be beneficial to the system. > > > If the IoTDB cluster is deployed on k8s, this parameter is more > > > indispensable, because k8s can dispatch another pod to replace this > > > OOM node rapidly. > > > Besides, i think we can add the usage of `-XX:+ExitOnOutOfMemoryError` > > > and `-XX:+HeapDumpOnOutOfMemoryError` in the user/DBA manual, which is > > > important to find the root cause of OOM. > > > > > > Best, > > > -- > > > Gaofei Cao > > > > > > Yuan Tian 于2023年11月13日周一 19:52写道： > > > > > > > > Hi all, > > > > > > > > Recently, we found in some real user cases that when OOM occurs in the > > > > DataNode process (although we should ensure that OOM does not happen, > > but > > > > we all know that bugs will always exist), some threads(e.g. rpc > > listening > > > > threads) may exit unexpectedly which may cause some strange things to > > > > happen. For example, if the heartbeat listening thread on the DataNode > > > > unexpectedly exits due to OOM, and then the OOM recovers on its own > > (some > > > > large queries end, or some compaction tasks end), but this thread will > > > > never exist again, causing the DataNode to remain in unknown state, > > > because > > > > the ConfigNode can no longer contact it via heartbeat. > > > > > > > > Therefore, we feel that OOM is a high-risk error, and we should let the > > > > process exit directly to avoid the loss of some key threads. > > > > > > > > And I did an experiment and found that -XX:+ExitOnOutOfMemoryError and > > > > -XX:+HeapDumpOnOutOfMemoryError do not conflict which means that we can > > > > keep both in jvm args and when OOM happens, it will firstly dump the > > heap > > > > memory and then exit. > > > > > > > > I've made this change in my pr( > > > https://github.com/apache/iotdb/pull/11531). > > > > > > > > What do you think? > > > > > > > > > > > > > > > > > > > > Best, > > > > -- > > > > Yuan Tian > > > > >
Re: Rollcall TsFile initial committers
Oh sorry, updated. Yuan Tian (jackiet...@apache.org) Chao Wang (wangchao...@apache.org) Christofer Dutz (cd...@apache.org) Jinrui Zhang (xingtan...@apache.org) Steve Yurong Su (r...@apache.org) Xinyu Tan (tanxi...@apache.org) Haonan Hou (hao...@apache.org) Gaofei Cao (gaogao...@apache.org) Jialin Qiao (qiaojia...@apache.org) Kun Liu (liu...@apache.org) Houliang Qi (neuyi...@apache.org) Xiangdong Huang (h...@apache.org) Chao Wang (ch...@apache.org) Jianmin Wang (jimw...@apache.org) Tian Jiang (jiangt...@apache.org） Xinyi Zhao (zhaoxi...@apache.org) Shuo Zhang (shuozh...@apache.org) Dawei Liu(li...@apache.org) Thanks, — Jialin Qiao Apache IoTDB PMC Haonan Hou 于2023年11月7日周二 09:38写道： > > Hi Jialin, > > I just find my name in list is not correct. It should be Haonan Hou, not > Haohan Hou. ;) > > Best, > Haonan > > > > On Nov 7, 2023, at 9:31 AM, Jialin Qiao wrote: > > > > Hi, > > > > Welcome Dawei! We are still on naming search stage. > > The updated list: > > > > Yuan Tian (jackiet...@apache.org) > > Chao Wang (wangchao...@apache.org) > > Christofer Dutz (cd...@apache.org) > > Jinrui Zhang (xingtan...@apache.org) > > Steve Yurong Su (r...@apache.org) > > Xinyu Tan (tanxi...@apache.org) > > Haohan Hou (hao...@apache.org) > > Gaofei Cao (gaogao...@apache.org) > > Jialin Qiao (qiaojia...@apache.org) > > Kun Liu (liu...@apache.org) > > Houliang Qi (neuyi...@apache.org) > > Xiangdong Huang (h...@apache.org) > > Chao Wang (ch...@apache.org) > > Jianmin Wang (jimw...@apache.org) > > Tian Jiang (jiangt...@apache.org） > > Xinyi Zhao (zhaoxi...@apache.org) > > Shuo Zhang (shuozh...@apache.org) > > Dawei Liu(li...@apache.org) > > > > Thanks, > > — > > Jialin Qiao > > Apache IoTDB PMC > > > > Dawei Liu 于2023年11月6日周一 21:09写道： > >> > >> Hi, > >> Recently changed computer equipment, didn't check my email, > >> and found out that something very important was missing... > >> Can i still participate? > >> > >> > >> > >> > >> > >> > >> > >> Thanks, > >> — > >> Dawei Liu > >> Apache IoTDB PMC > >> > >> > >> > >> > >> > >> > >> > >> > >> At 2023-10-23 14:04:25, "Jialin Qiao" wrote: > >>> Hi, > >>> > >>> Welcome Shuo! The updated list: > >>> > >>> Yuan Tian (jackiet...@apache.org) > >>> Chao Wang (wangchao...@apache.org) > >>> Christofer Dutz (cd...@apache.org) > >>> Jinrui Zhang (xingtan...@apache.org) > >>> Steve Yurong Su (r...@apache.org) > >>> Xinyu Tan (tanxi...@apache.org) > >>> Haohan Hou (hao...@apache.org) > >>> Gaofei Cao (gaogao...@apache.org) > >>> Jialin Qiao (qiaojia...@apache.org) > >>> Kun Liu (liu...@apache.org) > >>> Houliang Qi (neuyi...@apache.org) > >>> Xiangdong Huang (h...@apache.org) > >>> Chao Wang (ch...@apache.org) > >>> Jianmin Wang (jimw...@apache.org) > >>> Tian Jiang (jiangt...@apache.org） > >>> Xinyi Zhao (zhaoxi...@apache.org) > >>> Shuo Zhang (shuozh...@apache.org) > >>> > >>> Thanks, > >>> — > >>> Jialin Qiao > >>> Apache IoTDB PMC > >>> > >>> Shuo Zhang 于2023年10月23日周一 11:24写道： > >>> > >>>> > >>>> Hi Jialin, > >>>> > >>>> I participated in the R of the early version of TsFile with our early > >>>> teammates. Although my code contributions have lessened in recent years, > >>>> I have continued to champion the file layer with the entire DB system. > >>>> Going forward, I am happy to further promote TsFile and its development. > >>>> > >>>> If you find it beneficial, I would be delighted to become one of the > >>>> initial committers. Thanks a lot. > >>>> > >>>> Best Regards, > >>>> > >>>> > >>>> Shuo ZHANG > >>>> zhangshu...@qq.com > >>>> > >>>> From: Jialin Qiao > >>>> Date: 2023-10-15 23:22 > >>>> To: dev > >>>> Subject: Rollcall TsFile initial committers > >>>> Hi, > >>>> > >>>> To make TsFile a TLP, we need to affirm the initial committer first. > >>>> The criteria for becoming the initial committer of TsFile is > >>>> prelimilary defined as meeting the following two conditions > >>>> simultaneously. > >>>> > >>>> (1) Apache IoTDB PMC member > >>>> (2) Made contributions to the TsFile module > >>>> > >>>> If you have made significant contributions to the TsFile module, you > >>>> can also apply to become an initial committer. > >>>> > >>>> If you would like to be the initial committer of TsFile, please reply > >>>> to this email in 1 week :-) > >>>> > >>>> Thanks, > >>>> — > >>>> Jialin Qiao > >>>> Apache IoTDB PMC >
Re: Re: Rollcall TsFile initial committers
Hi, Welcome Dawei! We are still on naming search stage. The updated list: Yuan Tian (jackiet...@apache.org) Chao Wang (wangchao...@apache.org) Christofer Dutz (cd...@apache.org) Jinrui Zhang (xingtan...@apache.org) Steve Yurong Su (r...@apache.org) Xinyu Tan (tanxi...@apache.org) Haohan Hou (hao...@apache.org) Gaofei Cao (gaogao...@apache.org) Jialin Qiao (qiaojia...@apache.org) Kun Liu (liu...@apache.org) Houliang Qi (neuyi...@apache.org) Xiangdong Huang (h...@apache.org) Chao Wang (ch...@apache.org) Jianmin Wang (jimw...@apache.org) Tian Jiang (jiangt...@apache.org） Xinyi Zhao (zhaoxi...@apache.org) Shuo Zhang (shuozh...@apache.org) Dawei Liu(li...@apache.org) Thanks, — Jialin Qiao Apache IoTDB PMC Dawei Liu 于2023年11月6日周一 21:09写道： > > Hi, > Recently changed computer equipment, didn't check my email, > and found out that something very important was missing... > Can i still participate? > > > > > > > > Thanks, > — > Dawei Liu > Apache IoTDB PMC > > > > > > > > > At 2023-10-23 14:04:25, "Jialin Qiao" wrote: > >Hi, > > > >Welcome Shuo! The updated list: > > > >Yuan Tian (jackiet...@apache.org) > >Chao Wang (wangchao...@apache.org) > >Christofer Dutz (cd...@apache.org) > >Jinrui Zhang (xingtan...@apache.org) > >Steve Yurong Su (r...@apache.org) > >Xinyu Tan (tanxi...@apache.org) > >Haohan Hou (hao...@apache.org) > >Gaofei Cao (gaogao...@apache.org) > >Jialin Qiao (qiaojia...@apache.org) > >Kun Liu (liu...@apache.org) > >Houliang Qi (neuyi...@apache.org) > >Xiangdong Huang (h...@apache.org) > >Chao Wang (ch...@apache.org) > >Jianmin Wang (jimw...@apache.org) > >Tian Jiang (jiangt...@apache.org） > >Xinyi Zhao (zhaoxi...@apache.org) > >Shuo Zhang (shuozh...@apache.org) > > > >Thanks, > >— > >Jialin Qiao > >Apache IoTDB PMC > > > >Shuo Zhang 于2023年10月23日周一 11:24写道： > > > >> > >> Hi Jialin, > >> > >> I participated in the R of the early version of TsFile with our early > >> teammates. Although my code contributions have lessened in recent years, I > >> have continued to champion the file layer with the entire DB system. Going > >> forward, I am happy to further promote TsFile and its development. > >> > >> If you find it beneficial, I would be delighted to become one of the > >> initial committers. Thanks a lot. > >> > >> Best Regards, > >> > >> > >> Shuo ZHANG > >> zhangshu...@qq.com > >> > >> From: Jialin Qiao > >> Date: 2023-10-15 23:22 > >> To: dev > >> Subject: Rollcall TsFile initial committers > >> Hi, > >> > >> To make TsFile a TLP, we need to affirm the initial committer first. > >> The criteria for becoming the initial committer of TsFile is > >> prelimilary defined as meeting the following two conditions > >> simultaneously. > >> > >> (1) Apache IoTDB PMC member > >> (2) Made contributions to the TsFile module > >> > >> If you have made significant contributions to the TsFile module, you > >> can also apply to become an initial committer. > >> > >> If you would like to be the initial committer of TsFile, please reply > >> to this email in 1 week :-) > >> > >> Thanks, > >> — > >> Jialin Qiao > >> Apache IoTDB PMC
Re: Re: Suggestions for Enhancing the IoTDB Contribution Experience
Hi, Thanks for your feedback of this contribution! I support to give more attention for each PR and do more communication. Besides, if you have a PR need to be reviewed or just want to do a modification, welcome to discuss in the mail list. Thanks, — Jialin Qiao Apache IoTDB PMC 张 于2023年11月3日周五 18:25写道： > After reading PR#11429, upon comparing my PR, it does have more > advantageous aspects from which I have greatly benefited. However, I need > to point out something that I find 'unfair' in the process. > > *First of all*, my PR was committed on Oct 12, 2023, and no one followed > up on it before I asked. > > > *The second key point *is that no one gave useful advice on where I > should make modifications or the aspects I might have overlooked. Thus, I > didn't have the opportunity to make better revisions... > > At the same time, I must clarify that due to my limited familiarity with > IoTDB, the principle I followed during the modification process was 'to > avoid making significant changes to the existing logic as much as > possible.' Therefore, I attempted to make only necessary adjustments based > on the existing source code. This is evident to anyone who carefully reads > my commits. However, this might not be a good solution in open-source PRs. > > I acknowledge that my PR might be "worse code". Nevertheless, I still hope > that in future PR reviews for IoTDB, if possible, reviewers will carefully > review the code. Even if the PR is truly subpar, reviewers should provide > feedback. This would be incredibly helpful for the person because they > would know where their issues lie. > > The above is what I want to express. I hope that the "processing" will be > more warming in the future. > > Best regards > > > At 2023-11-01 16:01:16, "Xiangdong Huang" wrote: > >Hi, > > > >I read the discussion and all related issues/pull requests, and give > >some of my comments: > > > >It is appreciated for cqzhang's contribution to iotdb, especially for > >the detailed bug feedback. This is the best contribution to the open > >source community. > > > >Being a TLP open source project of Apache Software Foundation, keeping > >diversity is the community's goal. > >Therefore, attracting more and more contributors contributing to IoTDB > >is what we always want. > > > >I have to say comments like "this issue may not be suitable for a > >newer to our system due to communication costs of these complicated > >cases" is ABSOLUTLY incorrect. It is very very harmful for the > >community. > >"newer" does not mean he/she knows little than "older", and does not > >mean his/her solution is worse than "older". > >However, "meritocracy" still takes effect, because hearing a person > >whom has make some successful experience is usual correct. > > > >The only way we need to obey is: accepting better quality of codes, > >which we still need to improve. > >For example, I find cqzhang gave a solution like "using 12 months to > >replace 1 year", which is accpeted in the final PR. This is a kind of > >contribution. > >In the closed PR (#11171), the most important shortcoming is lack of UT and > >IT. > >In the merged PR(#11323), it also has drawbacks, e.g., > >"DateTimeUtils.convertDurationStrToLong()" function semantic is still > >incorrect if the input parameter value is "y". > > > >Considering making it easy to let more developers join us, I advocate that: > >1. For each public issue in Github/JIRA, claim you are working on it > >to let all others knowing that. If some other person think he/she > >want to take over, please also claims it asap, rather than submit PR > >directly. > >2. Even though, the cases that two PRs solve the same issue may still > >occur. In this case, "better code wins" and meritocracy are still the > >best way. > > > >Best, > > > >--- > >Xiangdong Huang > >School of Software, Tsinghua University > > > > > > > >Weihao Li <18110526...@163.com> 于2023年10月31日周二 15:11写道： > >> > >> Hi cqzhang7, > >> As you can see, `not supporting group by nature year` and `not supporting > >> mixed units in group by` are two separate issues. > >> 1. For the first issue, we have discussed in > >> https://github.com/apache/iotdb/pull/11309, and the final PR was appended > >> on the discuss. > >> 2. For the second issue, your general idea is right, but there are still > >&g
Re: [DISCUSS] Champion and PMC Chair of TsFile
Hi, It's my honor to be the PMC Chair of TsFile. I will do my best for TsFile project. Bests, — Jialin Qiao Apache IoTDB PMC Chao Wang 于2023年10月26日周四 16:28写道： > > +1 for Christofer Dutz to be the Champion > > > +1 for Jialin Qiao to be the PMC Chair > > > > > Thanks! > > > Chao Wang > > > On 10/26/2023 10:50，张硕 wrote： > +1 for Christofer Dutz to be the Champion > > > +1 for Jialin Qiao to be the PMC Chair > > > > > > > ---Original--- > From: "Houliang Qi" Date: Thu, Oct 26, 2023 10:41 AM > To: "dev@iotdb.apache.org" Subject: Re: [DISCUSS] Champion and PMC Chair of TsFile > > > > > +1 for Christofer Dutz to be the Champion. Jialin Qiao to be the pmc chair. > > > > > Replied Message > | From | Chao Wang | > | Date | 10/26/2023 10:33 | > | To | | > | Subject | Re: [DISCUSS] Champion and PMC Chair of TsFile | > +1 ， for Christofer Dutz to be the Champion.Jialin Qiao to be the pmc chair. > > On 2023/10/26 02:20:14 Xiangdong Huang wrote: > Hi, > > I vote for Christofer Dutz to be the Champion. I think he is the best > person for the role. > We can feel how he is professional in another email thread (discussion > about the proposal and resolution) :D > > As for the PMC Chair, I think Jialin Qiao is a good choice, he really did a > lot on the project, knows everything about the project, including the > design, the goal, and pay almost all of his time on the project. > > Best, > --- > Xiangdong Huang > > > Jialin Qiao 于2023年10月25日周三 15:54写道： > > Hi, > > To enter the Apache, we also need to define the project champion and PMC > chair. > > For Champion, I am honored to invite Chris to be the champion of the > TsFile project. > > For PMC, welcome to recommend yourself and others. > > Thanks, > — > Jialin Qiao > Apache IoTDB PMC
Re: [DISCUSS] Apache TsFile Proposal
Hi, Looks great! +1 for name search
—
Jialin Qiao
Apache IoTDB PMC
Christofer Dutz 于2023年10月26日周四 17:19写道：
>
> FYI,
>
> https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-214
>
> Chris
>
> Von: Xiangdong Huang
> Datum: Donnerstag, 26. Oktober 2023 um 04:23
> An: dev@iotdb.apache.org
> Betreff: Re: [DISCUSS] Apache TsFile Proposal
> Hi,
>
> The content looks fine. +1 for name-search.
> Best,
> ---
> Xiangdong Huang
>
>
>
> Christofer Dutz 于2023年10月25日周三 22:24写道：
>
> >
> > More feedback from the board was that the description is pretty long and
> > having it shorted would leave us more wiggle-room in the future.
> > Also, should we formally have passed a name-search … if you’re ok with
> > this, I would start the Name-Search.
> >
> > So that would result in the following resolution:
> >
> > WHEREAS, the Board of Directors deems it to be in the best interests
> > of
> >the Foundation and consistent with the Foundation's purpose to
> >establish a Project Management Committee charged with the creation
> > and
> >maintenance of open-source software, for distribution at no charge
> > to
> >the public, related to a columnar storage file format designed for
> > time
> >series data.
> >
> >NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee
> >(PMC), to be known as the "Apache TsFile Project", be and hereby is
> >established pursuant to Bylaws of the Foundation; and be it further
> >
> >RESOLVED, that the Apache TsFile be and hereby is responsible for
> >the creation and maintenance of software related to a columnar
> > storage
> >file format designed for time series data.
> >
> >RESOLVED, that the office of "Vice President, Apache TsFile" be and
> >hereby is created, the person holding such office to serve at the
> >direction of the Board of Directors as the chair of the Apache
> >TsFile Project, and to have primary responsibility for management of
> >the projects within the scope of responsibility of the Apache TsFile
> >Project; and be it further
> >
> >RESOLVED, that the persons listed immediately below be and hereby
> > are
> >appointed to serve as the initial members of the Apache TsFile
> >Project:
> >
> >- Yuan Tian (jackietien at apache dot org)
> >- Chao Wang (wangchao316 at apache dot org)
> >- Christofer Dutz (cdutz at apache dot org)
> >- Jinrui Zhang (xingtanzjr at apache dot org)
> >- Steve Yurong Su (rong at apache dot org)
> >- Xinyu Tan (tanxinyu at apache dot org)
> >- Haohan Hou (haonan at apache dot org)
> >- Gaofei Cao (gaogaofei at apache dot org)
> >- Jialin Qiao (qiaojialin at apache dot org)
> >- Kun Liu (liukun at apache dot org)
> >- Houliang Qi (neuyilan at apache dot org)
> >- Xiangdong Huang (hxd at apache dot org)
> >- Chao Wang (chaow at apache dot org)
> >- Jianmin Wang (jimwang at apache dot org)
> >- Tian Jiang (jiangtian at apache dot org）
> >- Xinyi Zhao (zhaoxinyi at apache dot org)
> >- Shuo Zhang (shuozhagn at apache dot org)
> >
> >NOW, THEREFORE, BE IT FURTHER RESOLVED, that {name of the chair} be
> >appointed to the office of Vice President, Apache TsFile, to serve
> > in
> >accordance with and subject to the direction of the Board of
> > Directors
> >and the Bylaws of the Foundation until death, resignation,
> > retirement,
> >removal or disqualification, or until a successor is appointed; and
> > be
> >it further
> >
> >RESOLVED, that the Apache TsFile Project be and hereby is tasked
> > with
> >the migration and rationalization of the Apache IoTDB TsFile
> > subproject;
> >and be it further
> >
> >RESOLVED, that all responsibilities pertaining to the Apache IoTDB
> >TsFile subproject encumbered upon the Apache IoTDB PMC are hereafter
> >discharged.
> >
> >
> >
> >
> >
> > Von: Christofer Dutz
> > Datum: Mittwoch, 25. Oktober 2023 um 15:03
> > An: dev@iotdb.apache.org
> > Betreff: AW: [DISCUSS] Apache TsFile Proposal
> > So, it seems that this document is based on an incubation proposal … sorry
&
Re: [DISCUSS] Apache TsFile Proposal
Hi, After TsFile is being accepted by Apache, I will create an TsFile-Rust repo. Thanks, — Jialin Qiao Apache IoTDB PMC Giorgio Zoppi 于2023年10月25日周三 15:46写道： > > Hello, > can you add in apache a initial ts-native for the rust project? Sorry for > being pedantic. > Best Regards, > Giorgio > > Il giorno mer 25 ott 2023 alle ore 05:04 Jialin Qiao > ha scritto: > > > Hi, > > > > The TsFile proposal is as follows, feel free to give advice :-) > > > > Abstract > > > > TsFile is a columnar storage file format designed for time series > > data, which supports efficient compression, high throughput of read > > and write, and compatibility with various frameworks, such as Spark > > and Flink. It is easy to integrate TsFile into IoT big data processing > > frameworks. > > > > Proposal > > > > TsFile is used for managing time series data. Although it is firstly > > used inside IoTDB, many users and companies use TsFile directly as a > > time series data management solution independently. Besides, there is > > a growing requirement for Multi-Language Support of TsFile > > implementations, such as C++, Go and Rust. > > > > Apache IoTDB community hereby submit this proposal of TsFile as an > > independent Apache project. This proposal outlines the key features > > and benefits of TsFile, along with the integration plan and the need > > for multi-language support. > > > > Background > > > > Time series data is becoming increasingly important in a wide range of > > applications, including IoT, intelligent control, finance, log > > analysis, and monitoring systems. > > TsFile has been developed by Apache IoTDB community in Java and is in > > IoTDB repository. Users could store time series data using TsFile, > > then read and analyze it in IoTDB, Spark and Flink. IoTDB could also > > generate TsFiles and syncronize TsFiles between two IoTDB instances. > > Furthermore, the demand for TsFile implementations in multiple > > programming languages has been growing, as it allows developers to > > leverage TsFile's capabilities in their preferred language. > > > > TsFile offers several distinctive features and benefits: > > Efficient Storage and Compression: TsFile employs advanced compression > > techniques to minimize storage requirements, resulting in reduced disk > > space consumption and improved system efficiency. > > Flexible Schema and Metadata Management: TsFile allows for directly > > write data without pre defining the schema, which is flexible for data > > aquisition. > > High Query Performance with time range: TsFile has indexed devices, > > sensors and time dimensions to accelerate query performance, enabling > > fast filtering and retrieval of time series data. > > Seamless Integration: TsFile is designed to seamlessly integrate with > > existing big data frameworks, such as Spark, Flink and Hadoop. > > > > > > Rationale > > > > Before using TsFile, there doesn't exist a file format for time > > series. The industry companies usually write time series data in > > various user-defined file format without unification, or use general > > columnar file format such as Parquet and ORC, which makes data > > collection and processing complicated without a standard. After > > TsFile, organizations could write data in TsFile inside end devices or > > gateway, then transfer TsFile to the cloud for unified management in > > IoTDB and other systems. In this way, we lower the network > > transmission and the computing resource consumption in the cloud. > > > > Initial Goals > > > > The initial goals include: > > > > Make TsFile an independent project that has its own SDK and > > documentation that is easier to use. > > Multi-Language Support of TsFile implementations apart from Java, such > > as C++, Go and Rust. > > Integrate more encoding and compression method in TsFile. > > More tools for TsFile: visualization tool, parsing tool, repair tool. > > > > > > Current Status > > > > Meritocracy > > > > We plan to invite the IoTDB committer to be the initial committer of > > TsFile. And we would like to follow ASF meritocratic principles and > > invite additional developers to participate. We will establish the > > documentation and encourage and monitor community participation so > > that privileges can be extended to those that contribute. > > > > Community > > > > The TsFile community is grown from the Apache IoTDB Community. The > > IoTDB co
[DISCUSS] Champion and PMC Chair of TsFile
Hi, To enter the Apache, we also need to define the project champion and PMC chair. For Champion, I am honored to invite Chris to be the champion of the TsFile project. For PMC, welcome to recommend yourself and others. Thanks, — Jialin Qiao Apache IoTDB PMC
[DISCUSS] Apache TsFile Proposal
Hi, The TsFile proposal is as follows, feel free to give advice :-) Abstract TsFile is a columnar storage file format designed for time series data, which supports efficient compression, high throughput of read and write, and compatibility with various frameworks, such as Spark and Flink. It is easy to integrate TsFile into IoT big data processing frameworks. Proposal TsFile is used for managing time series data. Although it is firstly used inside IoTDB, many users and companies use TsFile directly as a time series data management solution independently. Besides, there is a growing requirement for Multi-Language Support of TsFile implementations, such as C++, Go and Rust. Apache IoTDB community hereby submit this proposal of TsFile as an independent Apache project. This proposal outlines the key features and benefits of TsFile, along with the integration plan and the need for multi-language support. Background Time series data is becoming increasingly important in a wide range of applications, including IoT, intelligent control, finance, log analysis, and monitoring systems. TsFile has been developed by Apache IoTDB community in Java and is in IoTDB repository. Users could store time series data using TsFile, then read and analyze it in IoTDB, Spark and Flink. IoTDB could also generate TsFiles and syncronize TsFiles between two IoTDB instances. Furthermore, the demand for TsFile implementations in multiple programming languages has been growing, as it allows developers to leverage TsFile's capabilities in their preferred language. TsFile offers several distinctive features and benefits: Efficient Storage and Compression: TsFile employs advanced compression techniques to minimize storage requirements, resulting in reduced disk space consumption and improved system efficiency. Flexible Schema and Metadata Management: TsFile allows for directly write data without pre defining the schema, which is flexible for data aquisition. High Query Performance with time range: TsFile has indexed devices, sensors and time dimensions to accelerate query performance, enabling fast filtering and retrieval of time series data. Seamless Integration: TsFile is designed to seamlessly integrate with existing big data frameworks, such as Spark, Flink and Hadoop. Rationale Before using TsFile, there doesn't exist a file format for time series. The industry companies usually write time series data in various user-defined file format without unification, or use general columnar file format such as Parquet and ORC, which makes data collection and processing complicated without a standard. After TsFile, organizations could write data in TsFile inside end devices or gateway, then transfer TsFile to the cloud for unified management in IoTDB and other systems. In this way, we lower the network transmission and the computing resource consumption in the cloud. Initial Goals The initial goals include: Make TsFile an independent project that has its own SDK and documentation that is easier to use. Multi-Language Support of TsFile implementations apart from Java, such as C++, Go and Rust. Integrate more encoding and compression method in TsFile. More tools for TsFile: visualization tool, parsing tool, repair tool. Current Status Meritocracy We plan to invite the IoTDB committer to be the initial committer of TsFile. And we would like to follow ASF meritocratic principles and invite additional developers to participate. We will establish the documentation and encourage and monitor community participation so that privileges can be extended to those that contribute. Community The TsFile community is grown from the Apache IoTDB Community. The IoTDB community is introducing TsFile at many technical conferences. Next, we will build the mailing list for more convenience, broader communication and archived discussions. We are open to recruiting more developers from diverse backgrounds. Core Developers TsFile initial PMCs are from IoTDB community: Christofer Dutz, Xiangdong Huang, Jialin Qiao, Steve Yurong Su, Jinrui Zhang, Yuan Tian, Xinyu Tan, Haonan Hou, Gaofei Cao, Tian Jiang, Chao Wang(wangchao316), Chao Wang(mychaow), Houliang Qi, Kun Liu. These people has extensive experience in building database and data management system. Alignment The ASF is the natural choice to host the TsFile project as its goal of encouraging community-driven open-source projects fits with our vision for TsFile. Additionally, many other projects with which we are familiar with and expect TsFile to integrate with, such as Apache Spark, Apache Flink and Apache IoTDB are hosted by the ASF and we will benefit and provide benefits in close proximity to them. Known Risks Project Name TsFile project is used in IoTDB and other scenarios for over 7 years, its name is unique. Orphaned Products The core developers plan to work full time on the project. There is very little risk of TsFile being abandoned as it is part of Apache IoTDB's internal
Re: Rollcall TsFile initial committers
Hi, Welcome Shuo! The updated list: Yuan Tian (jackiet...@apache.org) Chao Wang (wangchao...@apache.org) Christofer Dutz (cd...@apache.org) Jinrui Zhang (xingtan...@apache.org) Steve Yurong Su (r...@apache.org) Xinyu Tan (tanxi...@apache.org) Haohan Hou (hao...@apache.org) Gaofei Cao (gaogao...@apache.org) Jialin Qiao (qiaojia...@apache.org) Kun Liu (liu...@apache.org) Houliang Qi (neuyi...@apache.org) Xiangdong Huang (h...@apache.org) Chao Wang (ch...@apache.org) Jianmin Wang (jimw...@apache.org) Tian Jiang (jiangt...@apache.org） Xinyi Zhao (zhaoxi...@apache.org) Shuo Zhang (shuozh...@apache.org) Thanks, — Jialin Qiao Apache IoTDB PMC Shuo Zhang 于2023年10月23日周一 11:24写道： > > Hi Jialin, > > I participated in the R of the early version of TsFile with our early > teammates. Although my code contributions have lessened in recent years, I > have continued to champion the file layer with the entire DB system. Going > forward, I am happy to further promote TsFile and its development. > > If you find it beneficial, I would be delighted to become one of the initial > committers. Thanks a lot. > > Best Regards, > > > Shuo ZHANG > zhangshu...@qq.com > > From: Jialin Qiao > Date: 2023-10-15 23:22 > To: dev > Subject: Rollcall TsFile initial committers > Hi, > > To make TsFile a TLP, we need to affirm the initial committer first. > The criteria for becoming the initial committer of TsFile is > prelimilary defined as meeting the following two conditions > simultaneously. > > (1) Apache IoTDB PMC member > (2) Made contributions to the TsFile module > > If you have made significant contributions to the TsFile module, you > can also apply to become an initial committer. > > If you would like to be the initial committer of TsFile, please reply > to this email in 1 week :-) > > Thanks, > — > Jialin Qiao > Apache IoTDB PMC
Re: Rollcall TsFile initial committers
Hi, Welcome! The updated list: Yuan Tian (jackiet...@apache.org) Chao Wang (wangchao...@apache.org) Christofer Dutz (cd...@apache.org) Jinrui Zhang (xingtan...@apache.org) Steve Yurong Su (r...@apache.org) Xinyu Tan (tanxi...@apache.org) Haohan Hou (hao...@apache.org) Gaofei Cao (gaogao...@apache.org) Jialin Qiao (qiaojia...@apache.org) Kun Liu (liu...@apache.org) Houliang Qi (neuyi...@apache.org) Xiangdong Huang (h...@apache.org) Chao Wang (ch...@apache.org) Jianmin Wang (jimw...@apache.org) Tian Jiang (jiangt...@apache.org） Xinyi Zhao (zhaoxi...@apache.org) Thanks — Jialin Qiao Apache IoTDB PMC Stefanie Zhao 于2023年10月23日周一 09:22写道： > > Hi Jialin, > > I’d also like to be the initial committer of TsFile project. I major > participated in the user interface, for example functions design and TsFile > user manual. > > Best, > Xinyi Zhao > > On 2023/10/15 15:22:11 Jialin Qiao wrote: > > Hi, > > > > To make TsFile a TLP, we need to affirm the initial committer first. > > The criteria for becoming the initial committer of TsFile is > > prelimilary defined as meeting the following two conditions > > simultaneously. > > > > (1) Apache IoTDB PMC member > > (2) Made contributions to the TsFile module > > > > If you have made significant contributions to the TsFile module, you > > can also apply to become an initial committer. > > > > If you would like to be the initial committer of TsFile, please reply > > to this email in 1 week :-) > > > > Thanks, > > — > > Jialin Qiao > > Apache IoTDB PMC > >
Re: Rollcall TsFile initial committers
Hi, Thanks for all of your contribution! Following our criterion, the Initial Committers are: Yuan Tian (jackiet...@apache.org) Chao Wang (wangchao...@apache.org) Christofer Dutz (cd...@apache.org) Jinrui Zhang (xingtan...@apache.org) Steve Yurong Su (r...@apache.org) Xinyu Tan (tanxi...@apache.org) Haohan Hou (hao...@apache.org) Gaofei Cao (gaogao...@apache.org) Jialin Qiao (qiaojia...@apache.org) Kun Liu (liu...@apache.org) Houliang Qi (neuyi...@apache.org) Xiangdong Huang (h...@apache.org) Chao Wang (ch...@apache.org) Jianmin Wang (jimw...@apache.org) Tian Jiang (jiangt...@apache.org） Thanks, — Jialin Qiao Apache IoTDB PMC Chao Wang 于2023年10月21日周六 23:58写道： > > Hi Jialin， > > > I also would like to be the initial committer for the TsFile project。 > > > I am familiar with the TsFile format. I have participated in the discussion > on the index organization design of version 0.11. This is also a relatively > large format adjustment for TsFile, and I have also optimized some issues > during tsfile sync. > > > TsFile now has so many features, including encoding, indexing, etc., and it > seems that it does hope to become a common format in the field of industrial > big data and contribute to the industrial field. I am currently also doing > some work on some OLTP and OLAP databases. I hope that I will have the > opportunity to promote TsFile to open up the data flow with other databases > and expand the scope of use of TsFile in the future. > > > > > Thanks! > > > Chao Wang > On 10/20/2023 23:14，Houliang Qi wrote： > Hi Jialin, > > I also would like to be the initial committer for the TsFile project。 > > I am familiar with the format of TsFile and have contributed features such as > cross multi time partition load TsFile. > > Currently, I am also working on OLAP, and some OLAP databases already support > reading file formats such as parquet and orc. I hope that TsFile can become a > top-level project in Apache, so that other OLAP databases can support reading > TsFile format and expand the usage scenarios of TsFile. I am willing to make > my contribution to this. > > Thanks, > Houliang > Replied Message > | From | Xiangdong Huang | > | Date | 10/20/2023 22:29 | > | To | | > | Subject | Re: Rollcall TsFile initial committers | > Hi Jialin, > > I would also like to be an initial committer for the TsFile project. > > Best, > --- > Xiangdong Huang > > > Kun Liu 于2023年10月20日周五 16:28写道： > > Hi jialin > > I also would like to be the initial committer for the TsFile project, and > participated the initial version of the TsFile format. > > I am familiar with the `write path` and Rust language, if user and > community need the API for Tsfile with Rust I can help to make it. > > Thanks, > Kun > > > > Jialin Qiao 于2023年10月18日周三 12:48写道： > > Hi, > > Glad to receive so much support! > I also would like to be the initial committer of TsFile. > I'm familiar with the tech part(format, read and write) of TsFile, > application scenarios and solutions with TsFile and IoTDB. > Looking forward to making TsFile the infrastructure of IoT. > > Thanks, > — > Jialin Qiao > Apache IoTDB PMC > > Gaofei Cao 于2023年10月18日周三 00:20写道： > > Hi Jialin, > > I would like to be the initial committer of TsFile. > I participated in the design and development of the first version of > TsFile, familiar with the query process, file structure and other > system integrations. > And now I'm participating in the work to push down value filter. I > wish I could bring more features to make TsFile a better file format. > > > Thanks > > Gaofei Cao > > Haonan Hou 于2023年10月17日周二 14:39写道： > > > Hi Jialin, > > I would like to be the initial committer of TsFile. > > I participated in the development of TsFile v2 for IoTDB 0.10.x and > 0.11.x and the update tool for v1 to v2, v2 to v3. > > Thanks > Haonan Hou > > On 2023/10/17 03:40:42 Xinyu Tan wrote: > Hi Jialin, > > I would like to be the initial committer of TsFile. > > I participated in the design and development of IoTDB 0.12 cluster > version combined with TsFile for incremental snapshot. > > Now the consensus layer is designing a consensus algorithm based on > TsFile synchronization, which is expected to make TsFile more general in > more scenarios involving CDC and consensus. > > Thanks > > Xinyu Tan > > On 2023/10/15 15:22:11 Jialin Qiao wrote: > Hi, > > To make TsFile a TLP, we need to affirm the initial committer > first. > The criteria for becoming the
Re: [DISCUSS] Drop Spark 2.11
+1, but adding 2.13 and dropping 2.11 should be at the same time, so that we have the spark connector all the time. — Jialin Qiao Apache IoTDB PMC Christofer Dutz 于2023年10月17日周二 15:36写道： > > Hi all, > > while working on the dependency cleanup, I noticed that we were stuck with > some pretty old spark versions in the spark_2.11 module. > The reason is that the scala 2.11 versions have been end of life for quite > some time. Spark is now generally working with scala 2.12 and since version > 3.2 also 2.13. > > We have one reported CVE that we actually can’t get rid of in the scala_2.11 > module, as there will be no release with a fix. > > I therefore propose we drop the 2.11 scala spark plugin and possibly add a > 2.13 version instead. > > What do you folks think? > > Chris >
Re: Change schema temlpate to device template
+1 — Jialin Qiao Apache IoTDB PMC Yuan Tian 于2023年10月20日周五 08:34写道： > > Hi all, > > We decided to change all `schema template` in sql syntax to `device > template`, because it's easier to understand for users. We will keep > `schema template` usage for compatibility, but all the tests and docs will > be changed to `device template`. > > > Best regards, > - > Yuan Tian
Re: Rollcall TsFile initial committers
Hi, Glad to receive so much support! I also would like to be the initial committer of TsFile. I'm familiar with the tech part(format, read and write) of TsFile, application scenarios and solutions with TsFile and IoTDB. Looking forward to making TsFile the infrastructure of IoT. Thanks, — Jialin Qiao Apache IoTDB PMC Gaofei Cao 于2023年10月18日周三 00:20写道： > > Hi Jialin, > > I would like to be the initial committer of TsFile. > I participated in the design and development of the first version of > TsFile, familiar with the query process, file structure and other > system integrations. > And now I'm participating in the work to push down value filter. I > wish I could bring more features to make TsFile a better file format. > > > Thanks > > Gaofei Cao > > Haonan Hou 于2023年10月17日周二 14:39写道： > > > > > > Hi Jialin, > > > > I would like to be the initial committer of TsFile. > > > > I participated in the development of TsFile v2 for IoTDB 0.10.x and 0.11.x > > and the update tool for v1 to v2, v2 to v3. > > > > Thanks > > Haonan Hou > > > > On 2023/10/17 03:40:42 Xinyu Tan wrote: > > > Hi Jialin, > > > > > > I would like to be the initial committer of TsFile. > > > > > > I participated in the design and development of IoTDB 0.12 cluster > > > version combined with TsFile for incremental snapshot. > > > > > > Now the consensus layer is designing a consensus algorithm based on > > > TsFile synchronization, which is expected to make TsFile more general in > > > more scenarios involving CDC and consensus. > > > > > > Thanks > > > > > > Xinyu Tan > > > > > > On 2023/10/15 15:22:11 Jialin Qiao wrote: > > > > Hi, > > > > > > > > To make TsFile a TLP, we need to affirm the initial committer first. > > > > The criteria for becoming the initial committer of TsFile is > > > > prelimilary defined as meeting the following two conditions > > > > simultaneously. > > > > > > > > (1) Apache IoTDB PMC member > > > > (2) Made contributions to the TsFile module > > > > > > > > If you have made significant contributions to the TsFile module, you > > > > can also apply to become an initial committer. > > > > > > > > If you would like to be the initial committer of TsFile, please reply > > > > to this email in 1 week :-) > > > > > > > > Thanks, > > > > — > > > > Jialin Qiao > > > > Apache IoTDB PMC > > > > > > >
Re: Dependency management PR ready for being merged
Merged, Good work! — Jialin Qiao Apache IoTDB PMC
Rollcall TsFile initial committers
Hi, To make TsFile a TLP, we need to affirm the initial committer first. The criteria for becoming the initial committer of TsFile is prelimilary defined as meeting the following two conditions simultaneously. (1) Apache IoTDB PMC member (2) Made contributions to the TsFile module If you have made significant contributions to the TsFile module, you can also apply to become an initial committer. If you would like to be the initial committer of TsFile, please reply to this email in 1 week :-) Thanks, — Jialin Qiao Apache IoTDB PMC
Re: [VOTE] Apache IoTDB 1.2.2 RC1 release
Hi, +1 (binding) The source release: apache headers [ok] signatures and hashes [ok] LICENSE and NOTICE [ok] no jar files [ok] could compile from source: ./mvnw.sh clean install [ok] The binary distribution: version number in CLI [ok] signatures and hashes [ok] start in mac, jdk8 [ok] statements executed successfully: [ok] SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); select ** from root; select ** from root align by device; Thanks, — Jialin Qiao Apache IoTDB PMC Yuan Tian 于2023年10月12日周四 20:52写道： > > Hi, > > +1 (binding) > > The source release: > apache headers [ok] > signatures and hashes [ok] > LICENSE and NOTICE [ok] > no jar files [ok] > could compile from source: ./mvnw.sh clean install [ok] > > The binary distribution: > version number in CLI [ok] > signatures and hashes [ok] > start in mac, jdk8 [ok] > statements executed successfully: [ok] > > SET STORAGE GROUP TO root.turbine; > CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; > insert into root.turbine.d1(timestamp,s0) values(1,1); > insert into root.turbine.d1(timestamp,s0) values(2,2); > insert into root.turbine.d1(timestamp,s0) values(3,3); > insert into root.turbine.d2(time,s0) values(now(),1); > insert into root.turbine.d2(time,s0) values(now(),2); > insert into root.turbine.d2(time,s0) values(now(),3); > select ** from root; > select ** from root align by device; > > > Best, > - > Yuan Tian
Re: [DISCUSS] Incubating TsFile as an independent Apache project?
Hi all, I'm glad that we have reached a consensus to donate TsFile as an independent Apache project. TsFile aims to be a universal file structure for managing time series data in IoT scenarios. So the next step is the actual process of driving it to be an Apache top-level project. Is the process the same as becoming an incubator project? Does anyone know? Thanks, — Jialin Qiao Apache IoTDB PMC Jialin Qiao 于2023年10月8日周日 13:34写道： > > Hi, > > Code is not the blocking thing, it could be done before release. > > Thanks, > ————— > Jialin Qiao > Apache IoTDB PMC > > Yuan Tian 于2023年10月8日周日 08:27写道： > > > > Hi, > > > > +1. > > > > However, before we do that, I think there are lots of work that we need to > > do, like extracting some read and write public interface for TsFile, > > otherwise it will be confusing for users and they don't know which > > methods to use. And with these interfaces, it will be beneficial to our > > subsequent iterations. > > > > Best, > > - > > Yuan Tian
Re: [DISCUSS] Incubating TsFile as an independent Apache project?
Hi, Code is not the blocking thing, it could be done before release. Thanks, — Jialin Qiao Apache IoTDB PMC Yuan Tian 于2023年10月8日周日 08:27写道： > > Hi, > > +1. > > However, before we do that, I think there are lots of work that we need to > do, like extracting some read and write public interface for TsFile, > otherwise it will be confusing for users and they don't know which > methods to use. And with these interfaces, it will be beneficial to our > subsequent iterations. > > Best, > - > Yuan Tian
Re: [DISCUSS] Incubating TsFile as an independent Apache project?
Hi, +1 for Apache TsFile. 1. Industrial time series data management is becoming more and more important, so a file format project for time series data is necessary. 2. When TsFile becomes an independent project, it could be developed and released without impact the upgrading of IoTDB. Thanks, — Jialin Qiao Apache IoTDB PMC 冯 庆新 于2023年10月7日周六 10:22写道： > > Hi Steve Su > > I think this is a great idea and I fully support it. > I believe Tsfile can generate greater influence and value. > > Thanks > B.R. > Qingxin Feng > > > 从 Windows 版邮件<https://go.microsoft.com/fwlink/?LinkId=550986>发送 > > 发件人: Steve Yurong Su<mailto:r...@apache.org> > 发送时间: 2023年10月7日 0:53 > 收件人: dev@iotdb.apache.org<mailto:dev@iotdb.apache.org> > 主题: [DISCUSS] Incubating TsFile as an independent Apache project? > > Hi folks, > > Recently, I've been thinking about something that has been on my mind: > I believe TsFile has the potential to develop as an independent Apache > project (note, not a subproject like apache-iotdb-tsfile), just like > ORC, Parquet, and Arrow. How about incubating TsFile as an independent > Apache project? > > The main reason I came up with this idea is because I realized that > TsFile is not just the data file format of IoTDB, but it can also be a > standard format in the IoT field. It provides excellent support for > time series data, offering various encoding and compression methods. > It can be easily self-parsed and its statistical information enables > fast data querying by big data software. I believe that every IoT > software could benefit directly by supporting TsFile as a storage > format. > > On the other hand, I hope to leverage the power of the community to > drive the development of the TsFile file format. Just imagine, TsFile > could have its own independent ecosystem: parsing tools in various > languages, connectors to various big data software, and so on. By > separating TsFile from IoTDB, we have the opportunity to attract more > community contributors and introduce more ideas, thus refining the > functionality and performance of TsFile. > > What do you think about this idea? > > In addition, how can TsFile, as a part of the current IoTDB project, > become an independent Apache project? Should it go through the Apache > Foundation incubation process again, or can it directly become a > top-level project? It may be too early to ask this question now, but I > would also like to hear your opinions. > > Thanks, > Steve Su >
Re: [VOTE] Apache IoTDB 1.2.1 RC1 release
Hi, +1 (binding) The source release: apache headers [ok] signatures and hashes [ok] LICENSE and NOTICE [ok] no jar files [ok] The binary distribution: version number in CLI [ok] signatures and hashes [ok] start in mac, jdk8 [ok] statements executed successfully: [ok] SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); select ** from root; Thanks, — Jialin Qiao Apache IoTDB PMC Yuan Tian 于2023年9月15日周五 18:14写道： > > Hi, > > +1 (binding) > > The source release: > apache headers [ok] > signatures and hashes [ok] > LICENSE and NOTICE [ok] > no jar files [ok] > could compile from source: ./mvnw.sh clean install [ok] > > The binary distribution: > version number in CLI [ok] > signatures and hashes [ok] > start in mac, jdk8 [ok] > statements executed successfully: [ok] > > SET STORAGE GROUP TO root.turbine; > CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; > insert into root.turbine.d1(timestamp,s0) values(1,1); > insert into root.turbine.d1(timestamp,s0) values(2,2); > insert into root.turbine.d1(timestamp,s0) values(3,3); > insert into root.turbine.d2(time,s0) values(now(),1); > insert into root.turbine.d2(time,s0) values(now(),2); > insert into root.turbine.d2(time,s0) values(now(),3); > select ** from root; > select ** from root align by device; > > > Best, > - > Yuan Tian
Re: iotdb quarter report (Q3)
Hi, Add some information of Project Activity: 1. Apache IoTDB ranking first in the benchANT Time Series: DevOps. https://benchant.com/ranking/database-ranking 2. Import a new feature pipe framework for lightweight streaming processing. This could be used for CDC (data synchronization from IoTDB to another system) and data subscription. 3. Extend the IT framwork to support multiple cluster instance in one test case, for testing data synchronization between two IoTDB cluster. 4. Optimized the load balance strategy of data insertion. Now we could spread the newly created data partition among the datanodes. Thanks, — Jialin Qiao Apache IoTDB PMC Yuan Tian 于2023年9月12日周二 09:10写道： > > Hi, > > +1. LGTM. > > > Best, > - > Yuan Tian
Re: [VOTE] Apache IoTDB 1.2.0 RC2 release
Hi, +1 (binding) The source release: apache headers [ok] signatures and hashes [ok] LICENSE and NOTICE [ok] no jar files [ok] could compile from source: ./mvnw.sh clean install [ok] The binary distribution: version number in CLI [ok] signatures and hashes [ok] start in mac, jdk8 [ok] statements executed successfully: [ok] SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select ** from root; Thanks, — Jialin Qiao Apache IoTDB PMC Haonan Hou 于2023年9月5日周二 10:50写道： > > Hi all, > > Apache IoTDB 1.2.0 has been staged under [2] and it’s time to vote > on accepting it for release. All Maven artifacts are available under [1]. > Voting will be open for 72hr. > A minimum of 3 binding +1 votes and more binding +1 than binding -1 > are required to pass. > > Release tag: v1.2.0 > Hash for the release tag: 5c1b68224b9bd77c3aab6e234de532a10c1edcae > > Before voting +1, PMC members are required to download > the signed source code package, compile it as provided, and test > the resulting executable on their own platform, along with also > verifying that the package meets the requirements of the ASF policy > on releases. [3] > > You can achieve the above by following [4]. > > [ ] +1 accept (indicate what you validated - e.g. performed the > non-RM items in [4]) > [ ] -1 reject (explanation required) > > > [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1130 > [2] https://dist.apache.org/repos/dist/dev/iotdb/1.2.0/rc2 > [3] https://www.apache.org/dev/release.html#approving-a-release > [4] > https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release > [5] https://github.com/apache/iotdb/blob/rc/1.2.0/RELEASE_NOTES.md > [6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS > > Best, > > Haonan Hou
Re: Re: [VOTE] Release IotDB-Tools: Thrift artifacts
+1 (binding) — Jialin Qiao Apache IoTDB PMC 张金瑞 <329920...@qq.com.invalid> 于2023年8月21日周一 20:07写道： > > +1 （binding） > > > Thanks, > Jinrui > > > > > > > > 原始邮件 > > > > 发件人："Yuan Tian"< jackietie...@gmail.com ; > > 发件时间：2023/8/21 20:01 > > 收件人："dev"< dev@iotdb.apache.org ; > > 主题：Re: [VOTE] Release IotDB-Tools: Thrift artifacts > > > +1 (binding) > > So glad to see that. > > > Best, > - > Yuan Tian > > On Mon, Aug 21, 2023 at 3:05 PM Christofer Dutz > wrote: > > Hi all, > > as you all might know, I have worked on streamlining our build for all > possible platforms. > In order to work around us having to re-deploy these artifacts every > Saturday, I have moved the artifacts out of the main repo into the > iotdb-bin-resources git repo. > > I’ve staged the artifacts at: > https://repository.apache.org/content/repositories/orgapacheiotdb-1129 > > Tag Name: iotdb-tools-thrift-v0.14.1.0 > > > https://github.com/apache/iotdb-bin-resources/releases/tag/iotdb-tools-thrift-v0.14.1.0 > > The source bundle generally consists of a pom.xml and an assembly xml … > so > it should be quick to review. > > > https://repository.apache.org/service/local/repositories/orgapacheiotdb-1129/content/org/apache/iotdb/tools/iotdb-tools-thrift/0.14.1.0/iotdb-tools-thrift-0.14.1.0-source-release.zip > > I’m intentionally not staging it on the release-svn, as nobody should be > required to consume this module but us ourselves. > > The vote stays open for 72hours … assuming enough support, I’ll then > finish changing the main repo to use these artifacts. > > > Chris >
Re: [VOTE] Apache IoTDB 1.1.2 RC1 release
Hi, +1 (binding) The source release: apache headers [ok] signatures and hashes [ok] LICENSE and NOTICE [ok] no jar files [ok] The binary distribution: version number in CLI [ok] signatures and hashes [ok] start in mac, jdk8 [ok] Thanks, — Jialin Qiao Apache IoTDB PMC Yuan Tian 于2023年7月26日周三 11:13写道： > > Hi, > > +1 (binding) > > The source release: > apache headers [ok] > signatures and hashes [ok] > LICENSE and NOTICE [ok] > no jar files [ok] > > The binary distribution: > version number in CLI [ok] > signatures and hashes [ok] > start in mac, jdk8 [ok] > statements executed successfully: [ok] > > SET STORAGE GROUP TO root.sg; > CREATE TIMESERIES root.sg.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; > insert into root.sg.d1(timestamp,s0) values(1,1); > insert into root.sg.d1(timestamp,s0) values(2,2); > insert into root.sg.d1(timestamp,s0) values(3,3); > select * from root.**; > select * from root.sg.** align by device; > > Best, > - > Yuan Tian
Re: Usage of Singleton pattern?
+1 for cleaning it — Jialin Qiao Apache IoTDB PMC Yuan Tian 于2023年7月14日周五 21:28写道： > > Sure! Do as you want! We would love to see these refactoring prs. > > > Best, > -- > Yuan Tian > > On Fri, Jul 14, 2023 at 9:20 PM Christofer Dutz > wrote: > > > > Aaah cool, > > > > and sorry for asking so much … much of IoTDBs history is not recorded and > > I’m just starting to get my hands dirty. > > I am afraid, that I’ll probably have more questions like this as they come > > up. > > > > But that means, if I see something that can be refactored quickly, I can > > create PRs for stuff like that? > > > > Just asking, cause I like to do simple maintenance tasks in order to think > > about other problems. > > In PLC4X I usually did cleanups regularly for such reasons. > > > > Chris > > > > > > Von: Yuan Tian > > Datum: Freitag, 14. Juli 2023 um 13:58 > > An: dev@iotdb.apache.org > > Betreff: Re: Usage of Singleton pattern? > > Hi Chris, > > > > Actually, most of the existing Singleton pattern Class is just for > > simplifying things during development. And we did run into trouble > > when writing some UTs(In such case, I will change related Singleton > > pattern Class ). > > But there are always higher priority tasks, so there is no time to > > change all these singletons. Good news is that we have strictly > > controlled the use of singletons in the new PRs, so there should be no > > more new ones. > > > > Best, > > > > Yuan Tian > > > > On Fri, Jul 14, 2023 at 7:14 PM Christofer Dutz > > wrote: > > > > > > Hi all, > > > > > > as I’m digging into IoTDB more and more, I noticed a quite excessive > > > usage of the Singleton pattern … some times even for things where it > > > doesn’t really make much sense. For example in SystemFileFactory, the > > > methods don’t really have much state and simply fire RuntimeExceptions if > > > fsType is initialized with HDFS. > > > > > > Singletons make it a bit tricky to run tests in parallel and to do > > > dynamic reloading of modules (like in OSGI). Most projects I’ve worked on > > > try to limit the use of singletons to an absolute minimum due to these > > > problems. > > > > > > I guess embedding IoTDB in an OSGI environment hasn’t been a use-case > > > yet, but I could imagine that random test failures while running tests in > > > parallel probably could have caused some issues. > > > > > > Was this a design decision that is based on some real-world necessity, or > > > for simplifying things during development? > > > I don’t want to put too much effort into proposing cleaning up things, if > > > there was a conscious decision to do things otherwise. > > > > > > Chris > > >
Re: [DISCUSS] Setup Sonarcloud for IoTDB?
Hi, We have the sonar cloud: https://sonarcloud.io/project/overview?id=apache_incubator-iotdb But it is only for committers: https://github.com/apache/iotdb/pull/10548 Thanks, — Jialin Qiao Apache IoTDB PMC Christofer Dutz 于2023年7月14日周五 21:24写道： > > Hi folks, > > I just had a look and noticed that we currently don’t have Sonarcloud > analysis setup. > At apache we have free access to using it for open-soucre and I’d like to > suggest to start using it. Some of the things it reports are definitely going > to be false positives, but it also spots a lot of things that could cause > problems before anyone has to file a bug report. > > What are your thoughts on this? > > Chris >
Re: Change LZ4 to the default compressor
+1 for option 1: Change LZ4 to the default compressor — Jialin Qiao Apache IoTDB PMC Yuan Tian 于2023年7月7日周五 11:57写道： > > Hi, > > vote for option 1: Change LZ4 to the default compressor, since it's a > compatible changes. > > > Best, > --- > Yuan Tian > > On Fri, Jul 7, 2023 at 11:10 AM yanze chen wrote: > > > > Hi all, > > > > In iot-benchmark experiment[1], we found that LZ4 outperformed SNAPPY in > > terms of (de)compression time and compression ratio in most scenarios. In > > the VLDB2022 paper[2], we also found that LZ4 performs no worse than SNAPPY > > on real-world datasets. > > > > Therefore, we consider changing the default compressor of IoTDB from SNAPPY > > to LZ4. How do you think about it? > > > > option 1: Change LZ4 to the default compressor > > option 2: Keep SNAAPY as the default compressor > > > > [1] https://apache-iotdb.feishu.cn/docx/ThmHdPKw9oJv95xvLhWco2WnnLh > > [2] https://sxsong.github.io/doc/22vldb-encoding.pdf > > > > -- > > Yanze Chen > > School of Software, Tsinghua University
New Repo: iotdb-docs
Hi, To manage our different versions of doc uniformly and clean the folder of code repo, I'd like to create a new repo: iotdb-docs. Then, we could put all versions(0.13, 1.0, 1.1, 1.2) of docs in the main branch of iotdb-docs, along with the sidebar. It will be easier to update the docs. What do you think? Thanks, — Jialin Qiao Apache IoTDB PMC
Re: Refactor the package structure of IoTDB
Hi Chris, This change originated from your advice in Slack :) Thanks, — Jialin Qiao Apache IoTDB PMC Christofer Dutz 于2023年6月30日周五 16:07写道： > > Hi Gaofei, > > I totally agree and support changing the package structure, as it was > something I stumbled over some time ago when I was trying to integrate IoTDB > into something. > > Chris > > > > Von: Gaofei Cao > Datum: Dienstag, 27. Juni 2023 um 15:19 > An: dev@iotdb.apache.org > Betreff: Refactor the package structure of IoTDB > Hi All, > > To make the the package structure of IoTDB more concise and tidy, we > have moved the package `antlr`, `confignode`, `consensus`, `metrics`, > `mlnode`, 'node-commons' and `server` to a new package `iotdb-core`. > > The package structure before changed: > Parent > ├──antrl > ├──confignode > ├──consensus > ├──iotdb-api > ├──iotdb-client > ├──iotdb-connector > ├──iotdb-protocol > ├──library-udf > ├──metrics > ├──mlnode > ├──node-commons > ├──server > ├──tsfile > > > The package structure after changed (master branch): > Parent > ├── iotdb-api > ├── iotdb-client > ├── iotdb-connector > ├── iotdb-core > └── antlr > └── confignode > └── consensus > └── datanode (sever package before changed) > └── metrics > └── mlnode > └── node-commons > └── tsfile > ├── iotdb-protocol > ├── library-udf > > > Thanks. > > Gaofei Cao > Apache IoTDB PMC
Re: [discuss] move jenkins build message to a new mailing list
+1 for option 1 Thanks, — Jialin Qiao Apache IoTDB PMC Haonan Hou 于2023年6月30日周五 17:31写道： > > +1 for option 1. > > Best, > Haonan Hou > > > On Jun 29, 2023, at 11:44 PM, Xiangdong Huang wrote: > > > > Hi all, > > > > How do you think about move jenkins's notification > > "build-failure"/"build-stable" to a new mailing list? > > > > Now, it is sent to dev@, and some people may think it is annoying.. > > > > So, how do you think about it? > > > > option 1: move to an existed mailing list: notifications@ (which is > > for jira issues now) > > option 2: create a new mailing list. > > option 3: keep current status > > > > Best, > > --- > > Xiangdong Huang >
Re: [New Feature] PB Tree for Massive Schema Management
Hi, Good feature! PB_Tree could be PBTree. Thanks, — Jialin Qiao Apache IoTDB PMC 周钰坤 于2023年6月19日周一 15:08写道： > > Hi, > > We're going to introduce PB Tree (Prefix-B+ Tree), a new SchemaEgnine > mode, in V1.2.0, which supports evict temporarily useless schema to > disk and load them back in need at runtime. With PB Tree, users will > not suffer from memory constraint for schema management any more, > especially in scenarios with massive time series. > > To enable PB Tree in IoTDB, the value of parameter named > schema_engine_mode in iotdb-commons.properties should be configured as > PB_Tree [1]. All the memory allocated for SchemaRegion will be used by > PB Tree. The allocation can be modified by changing the value of > parameters named storage_query_schema_consensus_free_memory_proportion > and schema_memory_proportion in iotdb-commons.properties. > > PB Tree is a new implementation of MTree in SchemaRegion, consisting > of a a PB Tree file and a prefix-tree-structured cache. The PB Tree > file manages the prefix tree structure on disk by storing pointer > pointing to first disk page storing children nodes in each node > record. B+ Tree is used to manage brother nodes for fast single child > node search. The cache is similar to the existing MTree implementation > in SchemaRegion. The main difference is that the children of a node > may not cached in memory, which should be read from disk by searching > specific B+ Tree located by pointer stored in node record and cached > in memory node object. > > Reference: > [1] > https://iotdb.apache.org/zh/UserGuide/V1.2.x/Reference/Common-Config-Manual.html#%E5%85%83%E6%95%B0%E6%8D%AE%E5%BC%95%E6%93%8E%E9%85%8D%E7%BD%AE > > Thanks. > > Yunkun Zhou > Apache IoTDB Committer
Release of 1.2.0
Hi, It's been about three months since we released 1.1.0. Considering there are some new features are developed these months. It may be time to release 1.2.0 in July. The new features I know contain PBTree of schema management, pipe function for data subscription of IoTDB, and some query functions. Please supplement the new features of 1.2.0. Thanks, — Jialin Qiao Apache IoTDB PMC
Re: [VOTE] Apache IoTDB 1.1.1 RC1 release
Hi, +1 (binding) The source release: apache headers [ok] signatures and hashes [ok] LICENSE and NOTICE [ok] no jar files [ok] The binary distribution: version number in CLI [ok] signatures and hashes [ok] start in mac, jdk8 [ok] statements executed successfully: [ok] SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select ** from root; Thanks, — Jialin Qiao On 2023/06/10 17:44:43 Haonan Hou wrote: > Hi all, > > Apache IoTDB 1.1.1 has been staged under [2] and it’s time to vote > on accepting it for release. All Maven artifacts are available under [1]. > Voting will be open for 72hr. > A minimum of 3 binding +1 votes and more binding +1 than binding -1 > are required to pass. > > Release tag: v1.1.1 > Hash for the release tag: d123b8e5ce8b23144107f41d1e1b115314d47ee0 > > Before voting +1, PMC members are required to download > the signed source code package, compile it as provided, and test > the resulting executable on their own platform, along with also > verifying that the package meets the requirements of the ASF policy > on releases. [3] > > You can achieve the above by following [4]. > > [ ] +1 accept (indicate what you validated - e.g. performed the > non-RM items in [4]) > [ ] -1 reject (explanation required) > > > [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1116 > [2] https://dist.apache.org/repos/dist/dev/iotdb/1.1.1/rc1 > [3] https://www.apache.org/dev/release.html#approving-a-release > [4] > https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release > [5] https://dist.apache.org/repos/dist/dev/iotdb/1.1.1/rc1/RELEASE_NOTES.md > [6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS > > Best, > > Haonan Hou >
Re: Contributing Code and Ongoing Support to the Newly Created C# Client Repository
Hi, Thanks for your contribution! — Jialin Qiao Apache IoTDB PMC ZHAN LU 于2023年5月27日周六 20:06写道： > > Dear friends, > > I am Zhan Lu, a first-year postgraduate student at the School of Software, > Tsinghua University, and a contributor to the Apache-IoTDB-Client-Csharp > <https://github.com/eedalong/Apache-IoTDB-Client-CSharp> repository. > > A new iotdb-client-csharp > <https://github.com/apache/iotdb-client-csharp> repository > has been created within the apache organization. Together with many friends > from the open-source community, we have contributed the code from our > earlier Apache-IoTDB-Client-Csharp repository to this new one, intending it > to be a strong foundation for future development and enhancements. > > We are committed to the ongoing contribution and maintenance of this > repository to facilitate continuous enhancements. > > I cordially invite all community members to utilize this new repository, > and sincerely welcome any feedback or suggestions that can help further its > development. > > Thank you for your time. Should you have any questions or need assistance > related to the new repository, please feel free to reach out. > > Best regards, > > Zhan Lu
new committer: Peichen Zhou
Hi, The Project Management Committee (PMC) for Apache IoTDB has invited Peichen Zhou to become a committer and we are pleased to announce that he has accepted. Being a committer enables easier contribution to the project since there is no need to go via the patch submission process. This should enable better productivity. You could get the write-access of our repository by: (1) Update your github id in: id.apache.org , also an email (anything to your apache email will forward to this email). (2) Accept the invitation in github to be a member of the apache group. (3) Link your accounts via: git.apache.org (To lighten the third option MFA Status, you need to download Google Authenticator (Google 身份验证器) in your mobile) (4）Sign in github by Google Authenticator (5) Clone repository with ssh. Welcome, Peichen! Yours, The Apache IoTDB PMC
[Result][VOTE] New Repo: iotdb-client-csharp
Hi, This vote [1] has lasted for 72 hours, with 3 +1 PMC votes and total 5 +1 votes, this VOTE passes! +1 votes: gitdata2 Houliang Qi Qingxin Feng Chao Wang Haonan Hou [1] https://lists.apache.org/thread/zb3wm5gz7cqxqcclsyqxxhygx643bkpr Thanks, — Jialin Qiao Apache IoTDB PMC
[VOTE] New Repo: iotdb-client-csharp
Hi, Currently, there are two Repositories of IoTDB Csharp client. eedalong/Apache-IoTDB-Client-CSharp eedalong/Apache-IoTDB-Client-CSharp-UseCase I prefer to call for a vote for creating a new repo iotdb-client-csharp for the csharp client. Please vote accordingly: [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove with the reason The vote is open for the next 72 hours and passes if at least three +1 votes and more +1 votes than -1 votes. Best, — Jialin Qiao Apache IoTDB PMC
Re: Multiple bug trackers?
Hi, GitHub is usually used by users. Developers are all using Jira now, moving to GitHub issues is not easy... We may need to discuss the benefit of moving to github. Thanks, — Jialin Qiao Apache IoTDB PMC Christofer Dutz 于2023年5月12日周五 14:57写道： > > Hi folks, > > just one question ... is there a reason we’re supporting both github issues > as well as Jira for Issue reporting? Wouldn’t it make more sense to decide on > one and go with that? (I would definitely recommend to move to GH Issues) > > Chris
CVE-2023-24831: Apache IoTDB grafana-connector Login Bypass Vulnerability
Severity: low Description: Improper Authentication vulnerability in Apache Software Foundation Apache IoTDB.This issue affects Apache IoTDB Grafana Connector: from 0.13.0 through 0.13.3. Attackers could login without authorization. This is fixed in 0.13.4. References: https://iotdb.apache.org/ https://iotdb.apache.org https://www.cve.org/CVERecord?id=CVE-2023-24831
CVE-2023-30771: Apache IoTDB Workbench: apache/iotdb-web-workbench: forge the JWTToken to access workbench
Severity: low Description: Incorrect Authorization vulnerability in Apache Software Foundation Apache IoTDB.This issue affects the iotdb-web-workbench component on 0.13.3. iotdb-web-workbench is an optional component of IoTDB, providing a web console of the database. This problem is fixed from version 0.13.4 of iotdb-web-workbench onwards. References: https://iotdb.apache.org https://www.cve.org/CVERecord?id=CVE-2023-30771
Re: [discuss] consider revert the feature of multi-tenancy
Hi, > I think we have reached a consensus from the discussion, change the name of > this feature to resource control, and continue to contribute to this feature. +1, Thanks for Houliang and Yuhua's contribution! We indeed need a resource control module to keep the system safe :) Best, — Jialin Qiao Apache IoTDB PMC Houliang Qi 于2023年4月11日周二 16:02写道： > > Hi, all > > > I think we have reached a consensus from the discussion, change the name of > this feature to resource control, and continue to contribute to this feature. > Thank you for your concern. > > > > > > Thanks, > --- > Houliang Qi > BONC, Ltd > > > Replied Message > | From | Xiangdong Huang | > | Date | 04/11/2023 15:39 | > | To | | > | Subject | Re: [discuss] consider revert the feature of multi-tenancy | > Hi Houliang, > > It makes no sense to refer Doris. Doris is not a lightweight db, and > edge side is never its goal. > > The topic of this discussion is whether to revert the feature of > multi-tenancy. > > I wonder why you fall into these words I think I have mentioned at > least twice (or maybe 3 times) that Jialin's suggestion is fine for > me. > > Best, > --- > Xiangdong Huang > School of Software, Tsinghua University > > > Houliang Qi 于2023年4月11日周二 15:05写道： > > Hi Jinrui, > > (Jinrui) From my perspective, Multi-tenancy is different from > resource-control and they are not the different term for same thing. > According to our implementation, current feature focus on the resource > control on users of one tenant rather than on different tenants. If we did > not reflect the wording `multi-tenancy` in the code, why do we use it on user > docs and PR's description ? > > Sorry, I am not agree with you, from my perspective, a user is a tenant, and > each tenant has different resources. This is also multi-tenancy. Even each > tenant can only have one db. In our current implementation, a user is a > tenant. > For doris, they also mention multi-tenancy, but it is limited user > resources.[1], the same as our current implementation. > For Spanner, a tenant can also have only one db. [2] > The reason why I think that both multi-tenancy and resource-control are > suitable for us is that what we are currently doing is to limit the functions > of users or db resources. > On this point, I agree with Wang Chao's point of view. > > As for whether the multi-tenant function you mentioned affects the > positioning of IoTDB, I don't think it is accurate. I personally think that > the multi-tenant function is a term for resource isolation technology and > will not affect the positioning of IoTDB. I don't know how you define the > multi-tenant function. If it refers to the connection with the billing system > of the cloud service provider, it may be another form. This discussion will > not continue to discuss multi-tenancy. > > > > (Jinrui) REVERT does not mean REJECT. It is only a quick way to keep the code > more reliable before we reach the same page. And furthermore, I don't think > it is harmful or discouraging and it is only a regular way we use to replace > hot-fix. > (Jinrui) The reviewers may be confused by the PR's description and then focus > on whether `multi-tenant` should be integrated in current development stage > of IoTDB. > > The topic of this discussion is whether to revert the feature of > multi-tenancy. I STRONGLY think that this PR does not violate the positioning > and future development of IOTDB, so I STRONGLY think that revert is not > needed, as this function is not enabled by default, and we are continuing > Iterate and refine this feature. Before the actual release, it is necessary > to consider some scenarios and do some testing. > > > > [1] https://doris.apache.org/docs/dev/admin-manual/multi-tenant/ > [2] > https://cloud.google.com/solutions/implementing-multi-tenancy-cloud-spanner > > > Thanks, > --- > Houliang Qi > BONC, Ltd > > > Replied Message > | From | Chao Wang | > | Date | 04/11/2023 13:42 | > | To | dev@iotdb.apache.org | > | Subject | Re: Re：Re: [discuss] consider revert the feature of multi-tenancy > | > Everyone's contribution counts. But what we are talking about is whether > `multi-tenancy` is suitable for current IoTDB's development. > From my perspective, Multi-tenancy is different from resource-control and > they are not the different term for same thing. According to our > implementation, current feature focus on the resource control on users of one > tenant rather than on
Re: [discuss] consider revert the feature of multi-tenancy
Hi, The following is my perspective. (1) The function name should be resource-control on the storage group level, rather than multi-tenancy. (2) This PR does not bring many side effects as it is disabled by default, so a revert is not a must. (3) The code may need to be improved, but it's ok as long as we keep optimizing. Thanks, — Jialin Qiao Apache IoTDB PMC Chao Wang 于2023年4月10日周一 19:14写道： > > Agree with Houliang's opinion. > > > Thanks! > > > Chao Wang > BONC ltd > On 4/10/2023 19:01，Houliang Qi wrote： > -1 > > First of all, thanks Xiangdong for pointing out IoTDB's Charter. > > "RESOLVED, that the Apache IoTDB Project be and hereby is > responsible for the creation and maintenance of software > related to an IoT native database with high performance > for data management and analysis, on the edge and the cloud." > > As the charter post, IoTDB can be deployed in the cloud, this is why we > deploy the multi-tenancy feature. > > The cloud can be a public or private cloud if we can deploy only one IoTDB > cluster, and manage multi databases and users with different resources, which > will simplify the maintenance. > > -> 1) how many side-effects the feature will bring; > > We have done some tests under[1], which says with 20 databases and 1 user > when we set `quota_enable` to true to enable the multi-tenancy feature, the > write performance is only slowed down 1.75%, the read latency has not much > difference, we will do more tests to show the side-effects in the feature. > > -> 2) how to reduce the effect when IoTDB is deployed on the edge. > > We supply one switch about this feature, called `quota_enable`, by default > this value is false, so it has no effect when IoTDB is deployed on the edge. > This also answers Jinrui's doubt. > > -> 3) some checks failed on WinOS, are they irrelevant? > > No, I think they are not irrelevant, the false check message is about the > Compaction module, and > I see the former pr[2][3] which have been merged 4 days ago has the same > issue, so I suspect that the compaction module has occasional bugs > > -> 4) The feature SHOULD be discussed carefully in the community, rather that > submit PRs and merged after some reviews. > > Besides the above, when we merge this pr, we posted the design in the > feishu[4] and discussed it online as least two times, and emailed and > discussed it with everyone[5], it has been passed 10 days. > > > The IoTDB community is open and different opinions are welcome. After all, we > all have the same original intention of wanting IoTDB's features to be more > diverse. > > [1] https://apache-iotdb.feishu.cn/docx/DbqCd8t3EoxlCFx1yYicd9N4n4s > [2] https://github.com/apache/iotdb/actions/runs/4625220921/jobs/8181102446 > [3] https://github.com/apache/iotdb/actions/runs/4531046594/jobs/7980725316 > [4] https://apache-iotdb.feishu.cn/docx/doxcnKOYKDmJ40FpVnVsPMd3nTg > [5] https://lists.apache.org/thread/y6dqcm2o7qk0nbkllb61bp8cv6d3m1h7 > > > > > > Thanks, > --- > Houliang Qi > BONC, Ltd > > > Replied Message > | From | 张金瑞<329920...@qq.com.INVALID> | > | Date | 04/10/2023 15:03 | > | To | dev | > | Subject | Re：[discuss] consider revert the feature of multi-tenancy | > +1, > > > Agree with Xiangdong's opinion. > And on the other hand, checking this PR's side effects may take lot of > time and during this period, there may be lots of users using latest > code to deploy/upgrade their systems. So the best practice is reverting this > PR until the side-effect is eliminated > > > > Thanks, > Zhang Jinrui，Apache IoTDB PMC > > > > Original > > > > From:"Xiangdong Huang"< saint...@gmail.com ; > > Date:2023/4/10 10:05 > > To:"dev"< dev@iotdb.apache.org ; > > Subject:[discuss] consider revert the feature of multi-tenancy > > > Hi all, > > I see the multi-tenancy feature is merged, and several committers made > a lot of contributions on that. > > As multi-tenancy is quite a big feature, which may change IoTDB's > position. The feature SHOULD be discussed carefully in the community, > rather that submit PRs and merged after some reviews. > > Therefore, I call to revert the PR and discuss ASAP about the feature > after that. > > At least, the proposer need to answer the following questions, > 1) how many side-effect the feature will bring; > 2) how to reduce the effect when IoTDB is deployed on the edge. > 3) some checks failed on WinOS, are they irrelevant? > > I don't mean of rejecting any big contribution to IoTDB or harming the > community's
Re: [VOTE] Apache IoTDB 1.1.0 RC2 release
Hi, +1 (binding) The source release: apache headers [ok] signatures and hashes [ok] LICENSE and NOTICE [ok] no jar files [ok] The binary distribution: version number in CLI [ok] signatures and hashes [ok] start in mac, jdk-17 [ok] statements executed successfully: [ok] insert into root.db.d1(time,s0) values(1,1); insert into root.db.d1(time,s0) values(2,2); insert into root.db.d1(time,s0) values(3,3); select * from root.db.**; Best, — Jialin Qiao Apache IoTDB PMC Haonan Hou 于2023年3月31日周五 11:44写道： > > Hi all, > > Apache IoTDB 1.1.0 has been staged under [2] and it’s time to vote > on accepting it for release. All Maven artifacts are available under [1]. > Voting will be open for 72hr. > A minimum of 3 binding +1 votes and more binding +1 than binding -1 > are required to pass. > > Release tag: v1.1.0 > Hash for the release tag: 807ec8b9e8006113d941ca7acd2c0a2f28829428 > > Before voting +1, PMC members are required to download > the signed source code package, compile it as provided, and test > the resulting executable on their own platform, along with also > verifying that the package meets the requirements of the ASF policy > on releases. [3] > > You can achieve the above by following [4]. > > [ ] +1 accept (indicate what you validated - e.g. performed the > non-RM items in [4]) > [ ] -1 reject (explanation required) > > > [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1115 > [2] https://dist.apache.org/repos/dist/dev/iotdb/1.1.0/rc2 > [3] https://www.apache.org/dev/release.html#approving-a-release > [4] > https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release > [5] https://dist.apache.org/repos/dist/dev/iotdb/1.1.0/rc2/RELEASE_NOTES.md > [6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS > > Best, > > Haonan Hou
Re: [VOTE] Apache IoTDB 1.1.0 RC1 release
Hi, +1 (binding) The source release: apache headers [ok] signatures and hashes [ok] LICENSE and NOTICE [ok] no jar files [ok] could compile from source: ./mvnw.sh clean install [ok] The binary distribution: version number in CLI [ok] signatures and hashes [ok] start in mac, jdk8 [ok] statements executed successfully: [ok] SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select ** from root; Thanks, Jialin Qiao On 2023/03/25 05:20:12 Haonan Hou wrote: > Hi all, > > Apache IoTDB 1.1.0 has been staged under [2] and it’s time to vote > on accepting it for release. All Maven artifacts are available under [1]. > Voting will be open for 72hr. > A minimum of 3 binding +1 votes and more binding +1 than binding -1 > are required to pass. > > Release tag: v1.1.0 > Hash for the release tag: 12e3f998667c72586139e023fe4521cb4db7c013 > > Before voting +1, PMC members are required to download > the signed source code package, compile it as provided, and test > the resulting executable on their own platform, along with also > verifying that the package meets the requirements of the ASF policy > on releases. [3] > > You can achieve the above by following [4]. > > [ ] +1 accept (indicate what you validated - e.g. performed the > non-RM items in [4]) > [ ] -1 reject (explanation required) > > > [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1114 > [2] https://dist.apache.org/repos/dist/dev/iotdb/1.1.0/rc1 > [3] https://www.apache.org/dev/release.html#approving-a-release > [4] > https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release > [5] https://dist.apache.org/repos/dist/dev/iotdb/1.1.0/rc1/RELEASE_NOTES.md > [6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS > > Best, > > Haonan Hou
Re: Request for translated names for group by clauses
Hi, +1 for 1. 差值分段 2. 条件分段 3. 会话分段 The group by series indeed confusing, we could use "group by filter" or "group by condition". Thanks, — Jialin Qiao Apache IoTDB PMC Eric Pai 于2023年2月24日周五 10:31写道： > > Hi， > > 1. 差值分段 > > 2. 条件分段 > > 3. 会话分段 > > I think the best choices are the direct translations. BTW, why we choose the > word 'series' in GROUP BY SERIES clause? It's more likely an aggregation > crossing different timeseries, like GROUP BY LEVEL and GROUP BY TAG. > > 发件人: ycy wi > 日期: 星期四, 2023年2月23日 18:04 > 收件人: dev@iotdb.apache.org > 主题: Request for translated names for group by clauses > > Hi, everyone! > > We are supporting more 'group by' clauses in aggregation query for IoTDB > which refers to some implements with new functions. So we want to ask for the > advice of translated name in Chinese for these new features according to the > functions and SQL of them. > > 1. Group By Variation > > Group-By-Variation wil set the first point in group as base point, then if > the difference between a new point and base point is small than or equal to > delta, this point will be put in the same group. (P.S. delta is the value > users set and the difference is calculated by the expression users give). It > can be described by the following picture: > [cid:ii_legxltnb3] > Some alternative translation in Chinese: > > * 事件分段 > * 差值分段 > * Any suggestions > 2. Group By Series > Group-By-Series supports grouping the data points by the condition expression > users give (the return type of condition expression needs to be boolean). The > continuous data points which satisfy the condition expression will be put in > the same group, the data points don't satisfy the condition will be skipped > and won't be calculated by aggregation operator. > Some alternative translation in Chinese: > > * 条件分段 > * Any suggestions > 3. Group By Session > Group-By-Session will compute the difference between the value of time in > continuous data points. A timeInterval is set by users to indicate the max > gap of time in one group between continuous data points. It can be described > in the following: > [cid:ii_legvj27g1] > The alternative translation in Chinese: > > * 会话分段 > * 时间差值分段 > * Any suggestions > > You can see more details and examples in > https://iotdb.apache.org/UserGuide/Master/Query-Data/Group-By.html<https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fiotdb.apache.org%2FUserGuide%2FMaster%2FQuery-Data%2FGroup-By.html=05%7C01%7C%7C7787a2489d0344286beb08db15855ebd%7C84df9e7fe9f640afb435%7C1%7C0%7C638127434791287241%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=NtoQ7ek7p7HXonZ837OkdW%2BajBAxL%2FnflUNYCCjQF8k%3D=0>. > > Reply 1,2,3 to represent each groupByClause and add your advice, if you > suggest "事件分段" for Group-By-Variation, you can reply > 1: "事件分段" >
Re: Remove UserGuide before 0.13 from website
Hi, You could find the history UserGuide in the "docs" folder of source code[1] [1] https://archive.apache.org/dist/iotdb/ Thanks, ————— Jialin Qiao Apache IoTDB PMC Trevor Hart 于2023年2月24日周五 01:15写道： > > +1 for having an archive available. I still use these docs and I am using > 0.12. > > > > Thanks > > Trevor Hart > > Ope Limited > > w: http://www.ope.nz/ > > m: +64212728039 > > > > > > > > > On Fri, 24 Feb 2023 06:10:08 +1300 Xiangdong Huang > wrote --- > > > > I think it is ok to remove old docs out of the source repo. > But is there a possible way to archive them? > > Best， > --- > Xiangdong Huang > School of Software, Tsinghua University > > 黄向东 > 清华大学 软件学院 > > 谭新宇 <mailto:1025599...@qq.com.invalid> 于2023年2月23日周四 20:34写道： > > > > Hi, > > > > +1 for removing old docs > > > > > > Thanks > > -- > > Xinyu Tan > > > > > > > 2023年2月23日 18:30，Jialin Qiao <mailto:qiaojia...@apache.org> 写道： > > > > > > Hi, > > > > > > There are 8 versions UserGuide on our website[1], the old versions are > > > hardly used anymore. > > > > > > I prefer to remove the history versions before 0.13 (not include). > > > > > > [1] https://iotdb.apache.org/ > > > > > > Thanks, > > > — > > > Jialin Qiao > > > Apache IoTDB PMC > >
Remove UserGuide before 0.13 from website
Hi, There are 8 versions UserGuide on our website[1], the old versions are hardly used anymore. I prefer to remove the history versions before 0.13 (not include). [1] https://iotdb.apache.org/ Thanks, — Jialin Qiao Apache IoTDB PMC
Re: [VOTE] Apache IoTDB 1.0.1 RC1 release
Hi, +1 (binding) The source release: LICENSE and NOTICE [ok] signatures and hashes [ok] All files have ASF header [ok] could compile from source: ./mvnw clean install [ok] The binary distribution: LICENSE and NOTICE [ok] signatures and hashes [ok] Could run with the following statements [ok] CREATE DATABASE root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select * from root.**; Thanks, — Jialin Qiao Apache IoTDB PMC Gaofei Cao 于2023年2月17日周五 19:26写道： > > Hi all, > > Apache IoTDB 1.0.1 has been staged under [2] and it’s time to vote > on accepting it for release. All Maven artifacts are available under [1]. > Voting will be open for 72hr. > A minimum of 3 binding +1 votes and more binding +1 than binding -1 > are required to pass. > > Release tag: v1.0.1 > Hash for the release tag: ff6de906748d9ddc1a3b40a768c3de69fecc6733 > > Before voting +1, PMC members are required to download > the signed source code package, compile it as provided, and test > the resulting executable on their own platform, along with also > verifying that the package meets the requirements of the ASF policy > on releases. [3] > > You can achieve the above by following [4]. > > [ ] +1 accept (indicate what you validated - e.g. performed the > non-RM items in [4]) > [ ] -1 reject (explanation required) > > > [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1097 > [2] https://dist.apache.org/repos/dist/dev/iotdb/1.0.1/rc1 > [3] https://www.apache.org/dev/release.html#approving-a-release > [4] > https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release > [5] https://dist.apache.org/repos/dist/dev/iotdb/1.0.1/rc1/RELEASE_NOTES.md > [6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS > > Best, > > Gaofei Cao
Re: [VOTE] Apache IoTDB 0.13.4 RC1 release
Hi, +1 (binding) The source release: apache headers [ok] signatures and hashes [ok] LICENSE and NOTICE [ok] no jar files [ok] could compile from source: ./mvnw.sh clean install -DskipTests [ok] The binary distribution: version number in CLI [ok] signatures and hashes [ok] start in mac, jdk8 [ok] statements executed successfully: [ok] SET STORAGE GROUP TO root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select ** from root; Thanks, — Jialin Qiao School of Software, Tsinghua University 刘 旭鑫 于2023年2月7日周二 18:17写道： > > Hi all, > > Apache IoTDB 0.13.4 has been staged under [1] and it’s time to vote on > accepting it for release. All Maven artifacts are available under. Voting > will be open for 72hr. A minimum of 3 binding +1 votes and more binding +1 > than binding -1 are required to pass. > > Release tag: v0.13.4 > > Hash for the release tag: 12260bc6e1eadb8e417752e15cc2813750242ba1 > > Before voting +1, PMC members are required to download the signed source code > package, compile it as provided, and test the resulting executable on their > own platform, along with also verifying that the package meets the > requirements of the ASF policy on releases. [2] > > You can achieve the above by following [3]. > > [ ] +1 accept (indicate what you validated - e.g. performed the non-RM items > in [3]) > [ ] -1 reject (explanation required) > > > [1] https://dist.apache.org/repos/dist/dev/iotdb/0.13.4 > [2] https://www.apache.org/dev/release.html#approving-a-release > [3] > https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release > [4] https://dist.apache.org/repos/dist/dev/iotdb/0.13.4/rc1/RELEASE_NOTES.md > [5] https://dist.apache.org/repos/dist/dev/iotdb/KEYS
Re: About naming of VerticallyConcatOperator
+1 for Plan A: Rename it as HorizontallyConcatOperator. — Jialin Qiao Apache IoTDB PMC Xiangwei Wei 于2023年2月6日周一 14:35写道： > > VerticallyConcatOperator is used to horizontally concatenate TsBlocks with > the same timestamp column. For example, > TsBlock A is: > | Time | ColumnA | > |1| 1.0| > TsBlock B is: > | Time | ColumnB | > |1| 2.0| > After the VerticallyConcatOperator, the result will be: > | Time | ColumnA | ColumnB | > |1| 1.0| 2.0 | > > Literally, I think it's a horizontal concatenation. So I suggest rename it > as HorizontallyConcatOperator. > > However, I asked the author about the reason for this name. He said, "From > the view of a table, adding columns means vertical changes." > > What's your opinion? > > +1 for Plan A: Rename it as HorizontallyConcatOperator. > +1 for Plan B: Keep current name VerticallyConcatOperator. > > -- > Best, > Xiangwei Wei
[Result][Vote] Create a repo for iotdb-backup-tool
Hi, The Vote for Creating a repo for iotdb-backup-tool closes. The vote results is: Binding votes: 3 +1, NonBinding votes 8 +1. No +0 or -1 vote. The votes come from: Xiangdong Huang (binding) HW-Chao Wang (binding) Qingxin Feng Guanfei Guo Jeck1son2 Haizeiwang Ximing Gao Guanchu Shen Tao Yu Meng Li Jialin Qiao (binding) The vote passes. Thanks for your votes! https://lists.apache.org/thread/lvsmtz8vk28nxnn9dqzqp4ryxszlvwn0 Bests, — Jialin Qiao Apache IoTDB PMC
Re: [VOTE] Create a repo for iotdb-backup-tool
+1 — Jialin Qiao Apache IoTDB PMC limen...@live.cn 于2023年2月1日周三 19:19写道： > > +1 > > > > limen...@live.cn > > From: Jialin Qiao > Date: 2023-01-31 21:49 > To: dev > Subject: [VOTE] Create a repo for iotdb-backup-tool > Hi, > > Here is a tool for backup IoTDB data: > https://github.com/apache/iotdb/pull/7248 > > As iotdb-backup-tool could be used independently, it could be a separate > repository for users. > > So, I'd like to start a vote for creating a new Repo for iotdb-backup-tool. > > Please vote accordingly: > > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove with the reason > > The vote is open for the next 72 hours and passes if at least three +1 > votes and more +1 votes than -1 votes. > > Thanks, > — > Jialin Qiao > Apache IoTDB PMC
Re: Release of 0.13.4
Thank Xuxin! — Jialin Qiao Apache IoTDB PMC 刘 旭鑫 于2023年2月4日周六 21:21写道： > > I am glad to. It would be a great honor for me to be the release manager. > > - > THSS > Liu Xuxin > > ____ > 发件人: Jialin Qiao > 发送时间: 2023年2月4日 13:17 > 收件人: dev@iotdb.apache.org > 主题: Release of 0.13.4 > > Hi all, > > It has been over 3 months since we release 0.13.3. It's time to > release 0.13.4. > > Does anyone want to be the release manager? > > Thanks, > — > Jialin Qiao > Apache IoTDB PMC
Release of 0.13.4
Hi all, It has been over 3 months since we release 0.13.3. It's time to release 0.13.4. Does anyone want to be the release manager? Thanks, — Jialin Qiao Apache IoTDB PMC
Release of IoTDB 0.13.4
Hi all, It has been over 3 months since we release 0.13.3. It's time to release 0.13.4, does anyone want to be the release manager? Thanks, — Jialin Qiao Apache IoTDB PMC
[VOTE] Create a repo for iotdb-backup-tool
Hi, Here is a tool for backup IoTDB data: https://github.com/apache/iotdb/pull/7248 As iotdb-backup-tool could be used independently, it could be a separate repository for users. So, I'd like to start a vote for creating a new Repo for iotdb-backup-tool. Please vote accordingly: [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove with the reason The vote is open for the next 72 hours and passes if at least three +1 votes and more +1 votes than -1 votes. Thanks, — Jialin Qiao Apache IoTDB PMC
CVE-2023-24830: Apache IoTDB: apache/iotdb-web-workbench: create a user without authorization
Severity: low Description: Improper Authentication vulnerability in Apache Software Foundation Apache IoTDB.This issue affects Apache IoTDB: from 0.13.0 before 0.13.3. References: https://iotdb.apache.org https://www.cve.org/CVERecord?id=CVE-2023-24830
CVE-2023-24829: Apache IoTDB: apache/iotdb-web-workbench: forge the JWTToken to access workbench
Description: Incorrect Authorization vulnerability in Apache Software Foundation Apache IoTDB iotdb-web-workbench.This issue affects users' access to the system without authorization. This CVE is fixed in iotdb-web-workbench tag v0.13.3. References: https://iotdb.apache.org/ https://iotdb.apache.org https://www.cve.org/CVERecord?id=CVE-2023-24829
new committer: Rongzhao Chen
Hi, The Project Management Committee (PMC) for Apache IoTDB has invited Rongzhao Chen to become a committer and we are pleased to announce that he has accepted. Being a committer enables easier contribution to the project since there is no need to go via the patch submission process. This should enable better productivity. You could get the write-access of our repository by: (1) Update your github id in: id.apache.org , also an email (anything to your apache email will forward to this email). (2) Accept the invitation in github to be a member of the apache group. (3) Link your accounts via: git.apache.org (To lighten the third option MFA Status, you need to download Google Authenticator (Google 身份验证器) in your mobile) (4）Sign in github by Google Authenticator (5) Clone repository with ssh. Welcome, Yongzao! Yours, The Apache IoTDB PMC
new committer: Haiming Zhu
Hi, The Project Management Committee (PMC) for Apache IoTDB has invited Haiming Zhu to become a committer and we are pleased to announce that he has accepted. Being a committer enables easier contribution to the project since there is no need to go via the patch submission process. This should enable better productivity. You could get the write-access of our repository by: (1) Update your github id in: id.apache.org , also an email (anything to your apache email will forward to this email). (2) Accept the invitation in github to be a member of the apache group. (3) Link your accounts via: git.apache.org (To lighten the third option MFA Status, you need to download Google Authenticator (Google 身份验证器) in your mobile) (4）Sign in github by Google Authenticator (5) Clone repository with ssh. Welcome, Haiming! Yours, The Apache IoTDB PMC
Naming about entity to device
Hi, Since 0.12 or 0.13, we rename the device to entity in code, but leaving sql still using device, such as: show devices align by device In daily use, users and developers still use device instead of entity. Therefore, to conform to the usage habit, we'd better return back to device. Thanks, — Jialin Qiao Apache IoTDB PMC
Re: Change the default port of ConfigNode and DataNode
+1, 107 looks like IoT :) — Jialin Qiao Apache IoTDB PMC Gaofei Cao 于2022年12月20日周二 10:49写道： > > dn_rpc_port, 6667 => 6667 > dn_internal_port, 9003 => 10710 > dn_mpp_data_exchange_port, 8777 => 10720 > dn_schema_region_consensus_port, 50010 => 10730 > dn_data_region_consensus_port, 40010 => 10740 > cn_internal_port, 22277 => 10750 > cn_consensus_port, 22278 => 10760 > > Thanks, > --- > Gaofei Cao > > Gaofei Cao 于2022年12月20日周二 10:35写道： > > > > The original port of `dn_schema_region_consensus_port` in DataNode is > > 50010, which may be conflict with HDFS system. > > > > And the original used ports are 6667、9003、8777、50010、40010、22277 and > > 22278, which are not in a regular range, so we want to change the > > default used ports as the picture below. > > > > What do your opinion about the port changes? > > > > > > Thanks, > > --- > > Gaofei Cao
new committer: Hongyin Zhang
Hi, The Project Management Committee (PMC) for Apache IoTDB has invited Hongyin Zhang to become a committer and we are pleased to announce that he has accepted. Being a committer enables easier contribution to the project since there is no need to go via the patch submission process. This should enable better productivity. You could get the write-access of our repository by: (1) Update your github id in: id.apache.org , also an email (anything to your apache email will forward to this email). (2) Accept the invitation in github to be a member of the apache group. (3) Link your accounts via: git.apache.org (To lighten the third option MFA Status, you need to download Google Authenticator (Google 身份验证器) in your mobile) (4）Sign in github by Google Authenticator (5) Clone repository with ssh. Welcome, Hongyin! Yours, The Apache IoTDB PMC
new committer: Qingxin Feng
The Project Management Committee (PMC) for Apache IoTDB has invited Qingxin Feng to become a committer and we are pleased to announce that he has accepted. Being a committer enables easier contribution to the project since there is no need to go via the patch submission process. This should enable better productivity. You could get the write-access of our repository by: (1) Update your github id in: id.apache.org , also an email (anything to your apache email will forward to this email). (2) Accept the invitation in github to be a member of the apache group. (3) Link your accounts via: git.apache.org (To lighten the third option MFA Status, you need to download Google Authenticator (Google 身份验证器) in your mobile) (4）Sign in github by Google Authenticator (5) Clone repository with ssh. Welcome, Qingxin! Yours, The Apache IoTDB PMC
new PMC: Chao Wang
Hi, The Project Management Committee (PMC) for Apache IoTDB has invited Chao Wang to become a PMC and we are pleased to announce that he has accepted. Welcome, Chao Wang! Yours, The Apache IoTDB PMC
Re: [ANNOUNCE] Apache IoTDB 1.0.0 released
Congratulations! — Jialin Qiao Apache IoTDB PMC HW-Chao Wang <576749...@qq.com.invalid> 于2022年12月5日周一 22:27写道： > > Congratulations, we are looking forward to this version！ > > > > > ---Original--- > From: "Haonan Hou" Date: Mon, Dec 5, 2022 20:46 PM > To: "annou...@apache.org" Cc: "dev@iotdb.apache.org" Subject: [ANNOUNCE] Apache IoTDB 1.0.0 released > > > The Apache IoTDB team is pleased to announce the release of Apache IoTDB > 1.0.0. > > Apache IoTDB (Database for Internet of Things) is an IoT native database > with high performance for data management and analysis, deployable on the > edge and the cloud. > > Apache IoTDB 1.0.0 version has a new architecture that supports standalone > and cluster mode. > > > ## New Features > > * New architecture that supports standalone and cluster mode with two types > of nodes: ConfigNode, DataNode > * Support ConfigNode management: Start/Add, Stop, Remove > * Support DataNode management: Start/Add, Stop, Remove > * Support replication of ConfigNode, Schema and Data > * Support Consensus Protocol: Simple, IoT, Ratis > * Support Cluster management sql: show cluster, show regions > * Support administration in Cluster: User, Permission, Role management > * Support authorization when login and executing a command > * Support create/show/count/delete database > * Support show/count devices > * Support create/show/count timeseries > * Support schema template management > * Support MPP(massively parallel process) framework in cluster > * Support insertion, deletion and all query types in Cluster > * Support CSV import/export tools > * Support TsFile import/export tools > * Support Sync TsFile from an IoTDB with one replica to another with any > replica number > * Support UDF framework in Cluster > * Support new UDF 'change_points > * Support stateful, stateless Trigger in Cluster > * Support Select into in Cluster > * Support Continuous Query in Cluster > * Support flush on local/cluster > * Support clear cache on local/cluster > * Support metric for DataNode and ConfigNode with output to IoTDB, > Prometheus, and JMX > * Support DBAPI in python client > * Support RestApi, MQTT for cluster > * Support having clause and between expression in query > * Support order by timeseries in last query > * Support hot configuration of data_dirs > > The full release note is available at: > https://raw.githubusercontent.com/apache/iotdb/master/RELEASE_NOTES.md > > The release is available for download at: > http://iotdb.apache.org/Download > > Maven artifacts for JDBC driver, session SDK, TsFile SDK, Spark-connector, > Hadoop-connector, Hive-connector and Flink-connector can be found at: > https://search.maven.org/search?q=3Dg:org.apache.iotdb > > > Regards, > The Apache IoTDB team
Development after releasing 1.0.0
Hi, Since we have checkout the rel/1.0 branch. Bug-fix pr goes to both rel/1.0 and master. New features only go to master. Branch management could refer to : https://apache-iotdb.feishu.cn/docx/MKHpdb7ghoEAfFxAQO3cR1zSnGf Thanks, — Jialin Qiao Apache IoTDB PMC
Re: [VOTE] Apache IoTDB 1.0.0 RC5 release
Hi, +1 (binding) The source release: apache headers [ok] signatures and hashes [ok] LICENSE and NOTICE [ok] no jar files [ok] The binary distribution: version number in CLI [ok] signatures and hashes [ok] start in mac, jdk8 [ok] statements executed successfully: [ok] create database root.turbine; CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA; insert into root.turbine.d1(timestamp,s0) values(1,1); insert into root.turbine.d1(timestamp,s0) values(2,2); insert into root.turbine.d1(timestamp,s0) values(3,3); select ** from root; Thanks, — Jialin Qiao Apache IoTDB PMC Haonan Hou 于2022年12月2日周五 17:22写道： > > Hi all, > > Apache IoTDB 1.0.0 version has a new architecture that supports standalone > and cluster mode. > > Apache IoTDB 1.0.0 has been staged under [2] and it’s time to vote > on accepting it for release. All Maven artifacts are available under [1]. > Voting will be open for 72hr. > A minimum of 3 binding +1 votes and more binding +1 than binding -1 > are required to pass. > > Release tag: v1.0.0 > Hash for the release tag: fbbca3ffa3d54b2757c22502f7211b5a2be72c96 > > Before voting +1, PMC members are required to download > the signed source code package, compile it as provided, and test > the resulting executable on their own platform, along with also > verifying that the package meets the requirements of the ASF policy > on releases. [3] > > You can achieve the above by following [4]. > > [ ] +1 accept (indicate what you validated - e.g. performed the non-RM items > in [4]) > [ ] -1 reject (explanation required) > > > [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1093 > [2] https://dist.apache.org/repos/dist/dev/iotdb/1.0.0/rc5 > [3] https://www.apache.org/dev/release.html#approving-a-release > [4] > https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release > [5] https://dist.apache.org/repos/dist/dev/iotdb/1.0.0/rc5/RELEASE_NOTES.md > [6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS > > Best, > > Haonan Hou
Re: [VOTE] Apache IoTDB 1.0.0 RC2 release
Sorry for -1 again RC2 also has NPE in cross space compaction, which is fixed in https://github.com/apache/iotdb/pull/8277 We could start RC3 Jialin Qiao On 2022/11/30 17:23:51 Haonan Hou wrote: > Hi all, > > Apache IoTDB 1.0.0 version has a new architecture that supports standalone > and cluster mode. > > Apache IoTDB 1.0.0 has been staged under [2] and it’s time to vote > on accepting it for release. All Maven artifacts are available under [1]. > Voting will be open for 72hr. > A minimum of 3 binding +1 votes and more binding +1 than binding -1 > are required to pass. > > Release tag: v1.0.0 > Hash for the release tag: 4d0b3af94edd57df9115dbf56487a906bee187ae > > Before voting +1, PMC members are required to download > the signed source code package, compile it as provided, and test > the resulting executable on their own platform, along with also > verifying that the package meets the requirements of the ASF policy > on releases. [3] > > You can achieve the above by following [4]. > > [ ] +1 accept (indicate what you validated - e.g. performed the non-RM items > in [4]) > [ ] -1 reject (explanation required) > > > [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1090 > [2] https://dist.apache.org/repos/dist/dev/iotdb/1.0.0/rc2 > [3] https://www.apache.org/dev/release.html#approving-a-release > [4] > https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release > [5] https://dist.apache.org/repos/dist/dev/iotdb/1.0.0/rc2/RELEASE_NOTES.md > [6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS > > Best, > > Haonan Hou
Re: [VOTE] Apache IoTDB 1.0.0 RC1 release
-1 Cross-space compaction is an important feature. but it is not enabled due to IOTDB-5092 in RC1. Currently, this is fixed and we add more checks. Thanks, Jialin Qiao On 2022/11/29 20:25:55 Haonan Hou wrote: > Hi all, > > Apache IoTDB 1.0.0 version has a new architecture that supports standalone > and cluster mode. > > Apache IoTDB 1.0.0 has been staged under [2] and it’s time to vote > on accepting it for release. All Maven artifacts are available under [1]. > Voting will be open for 72hr. > A minimum of 3 binding +1 votes and more binding +1 than binding -1 > are required to pass. > > Release tag: 1.0.0 > Hash for the release tag: dffb1d9470597cbe6cfba9a3d0272ab4ddaf8f0f > > Before voting +1, PMC members are required to download > the signed source code package, compile it as provided, and test > the resulting executable on their own platform, along with also > verifying that the package meets the requirements of the ASF policy > on releases. [3] > > You can achieve the above by following [4]. > > [ ] +1 accept (indicate what you validated - e.g. performed the non-RM items > in [4]) > [ ] -1 reject (explanation required) > > > [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1089 > [2] https://dist.apache.org/repos/dist/dev/iotdb/1.0.0/rc1 > [3] https://www.apache.org/dev/release.html#approving-a-release > [4] > https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release > [5] https://dist.apache.org/repos/dist/dev/iotdb/1.0.0/rc1/RELEASE_NOTES.md > [6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS > > Best, > > Haonan Hou
Rename MultiLeaderConsensus to IoTConsensus
Hi, We plan to rename MultiLeaderConsensus to IoTConsensus 1. MultiLeader does not have many degrees of identification, and IoT is our characteristic. 2. The abbreviation of MultiLeader is ML, which is conflict with Machine Learning, which we may introduce in IoTDB in the future. Thanks, — Jialin Qiao Apache IoTDB PMC
Re: [VOTE] Usage of Apache IoTDB Logo as Stickers for A Meetup
+1 Glad to see a meetup in Germany~ — Jialin Qiao Apache IoTDB PMC Pengcheng Zheng 于2022年11月25日周五 00:11写道： > > Hi Xiangdong, > > Yes the agenda is planned as followed: > 18:00 - 18:10 Warm-up: Get to know each other > 18:10 - 18:15 Welcome > 18:15 - 19:00 Talk by Julian Feinauer: "Introduction to Apache IoTDB" > 19:15 - 20:00 Talk by Tim Mitsch: "Using IoTDB for high-volume energy data" > 20:00 - 21:00 Informal networking: Drink, food, snack & mingle > > More details of the meetup is to be found under [1]. > > > [1] https://www.meetup.com/de-DE/apache-iotdb-meetup/events/289831235/ > > Best regards, > Pengcheng > > Am Do., 24. Nov. 2022 um 15:43 Uhr schrieb Xiangdong Huang > : > > > > +1 for printing IoTDB stickers. > > BTW, is there a schedule? > > Best. > > --- > > Xiangdong Huang > > School of Software, Tsinghua University > > > > 黄向东 > > 清华大学 软件学院 > > > > Pengcheng Zheng 于2022年11月24日周四 01:21写道： > > > > > > Hi all, > > > > > > Since we have quite a few users and contributors of Apache IoTDB in > > > Germany now, we would like to gather up on 08 Dez 2022 as the "First > > > German Apache IoTDB User Group Meeting" and a party shortly before > > > Christmas :). Details of the meetup could be found under [1]. > > > > > > We have designed a roll-up of IoTDB for the meetup. The file could be > > > found under [2], and the size is 85*200cm with bleed. > > > > > > Additionally, Timecho as the sponsor for the meetup wants to print out > > > some of the Apache IoTDB stickers to hand out (just the IoTDB Logo), > > > for which the usage need to be voted. I will send a mail to > > > tradema...@apache.org if it is approved. > > > > > > Please vote accordingly: > > > [ ] +1 approve > > > [ ] +0 no opinion > > > [ ] -1 disapprove with the reason > > > > > > The vote is open for the next 72 hours and passes with at least three +1 > > > votes and more +1 votes than -1 votes. > > > > > > > > > [1] https://www.meetup.com/de-DE/apache-iotdb-meetup/events/289831235/ > > > [2] https://timechor.feishu.cn/file/boxcnVwjP4Ugd4EKTESW96BrHxh > > > > > > > > > Best regards, > > > > > > Pengcheng Zheng > > > Timecho
Re: 回复: Add constraint to the length of database name
Hi, +1 for omitting "root", but may be in 1.x version. Thanks, ————— Jialin Qiao Apache IoTDB PMC Chao Wang 于2022年11月18日周五 21:26写道： > > It's OK to use 64 bytes as the sg name, but root should not be included. That > is, root. should be with a length of no more than 64, not with > a length of no more than 60. > > > > > Then I raised another question, which can be discussed next time. > > > > > I think root is only useful for joining all sgs. In other scenarios, root is > not necessary. We should try to shield the root prefix for users, because it > is unnecessary to always input these characters. > > > > > Thanks! > > > Chao Wang > BONC ltd > ccgow...@163.com > 在2022年11月18日 19:58，周钰坤 写道： > We support suffix path, which will be concat to the prefix path in "from > clause", and full path, starting with "root", in "where" clause. The > "root" is a identifier for us to recognize which is a full path. > > Chao Wang 于2022年11月18日周五 18:24写道： > > +1， but why including root, i think the user could ignore the "root" when > we change sg to database. And we could ignore the root in the file path. > And, the sql select from root.dbname and select from dbname all are ok to > keep compatible. > > > > > Thanks! > > > Chao Wang > BONC ltd > ccgow...@163.com > 在2022年11月18日 18:02，冯 庆新 写道： > Agree with ‘add constraint to the length of database name’,but Can we > choose a value greater than 64? > > 发件人: Jialin Qiao<mailto:qiaojia...@apache.org> > 发送时间: 2022年11月18日 16:15 > 收件人: dev@iotdb.apache.org<mailto:dev@iotdb.apache.org> > 主题: Re: Add constraint to the length of database name > > +1 > — > Jialin Qiao > Apache IoTDB PMC > > 周钰坤 于2022年11月18日周五 15:44写道： > > Hi, > > We want to add constraint to the length of database name, as most > popular database systems have such constraint as well, for example the > length of database name in Mysql shall not exceed 64. Currently, the > maximum length of database name, including "root.", is *64* and it is > immutable. Such constraint can help avoid some bugs of database and region > management, since we use database names in the directory name, which shall > not exceed the max name length defined by file system. > > best regards > > Yukun Zhou, Tsinghua University > >
Re: Add constraint to the length of database name
+1 — Jialin Qiao Apache IoTDB PMC 周钰坤 于2022年11月18日周五 15:44写道： > > Hi, > > We want to add constraint to the length of database name, as most > popular database systems have such constraint as well, for example the > length of database name in Mysql shall not exceed 64. Currently, the > maximum length of database name, including "root.", is *64* and it is > immutable. Such constraint can help avoid some bugs of database and region > management, since we use database names in the directory name, which shall > not exceed the max name length defined by file system. > > best regards > > Yukun Zhou, Tsinghua University
Standardize TSStatusCode
Hi, Recently, I standardized the TSStatusCode, changed most values of the code, and supplement the UserGuide. When updating the TSStatusCode class, please remember to update the UserGuide. https://github.com/apache/iotdb/pull/8026 https://github.com/apache/iotdb/pull/8044 Thanks, — Jialin Qiao Apache IoTDB PMC
[RESULT][VOTE] Create a repo for TsFile-Viewer
Hi, This vote [1] has lasted for 72 hours, with 3 +1 PMC votes and No +0 or -1 votes, this VOTE passes! +1 Votes: Haonan Hou(binding) Jinrui Zhang(binding) Gaofei Cao(binding) Meng Li 海賊王ルフィ Chao Wang Guanfei Guo Liwen Fu Xin Hua jeck1son2 Tao Yu Hongshan Miao Guanchu Shen Thanks for your voting! [1] https://lists.apache.org/thread/qhwlms519wbot3bs3lowf2wqr8borz2v Bests, — Jialin Qiao Apache IoTDB PMC
Rename storage group to database and RPCAddress to ClientAddress
Hi, (1) In the coming 1.0 version, we prefer to let users only set one storage group, which is already renamed to the database. However, in some query result, such as show timeseries, show ttl, we still return a column named storage group. To be compatible with 0.13 version, I prefer to add a parameter database_name=database/stoarage group to control the return column name, default is database. (2) Rename the RPCAddress to ClientAddress, and keep RPCPort as usual. Thanks, — Jialin Qiao Apache IoTDB PMC
Re: [DISCUSS] Create a repo for TsFile-Viewer
Hi Chris, Thanks for pointing this out. We need to investigate how to solve this... Jialin Qiao Christofer Dutz 于2022年11月8日周二 16:45写道： > > Hi all > > I just had a look at what UI technology this PR is using, as for PLC4X we’re > currently looking for an alternative. > > Unfortunately, it seems to be using JavaFX. This wasn’t a problem in the > past, when JavaFX was part of the JDK and the GPL with Classpath Exception > was simple. > > However, I would like to point out that JavaFX is genereally GPL and can’t be > bundled with any binary distribution. > Users will have to manually install JavaFX and a “compile” dependency > unfortunately can’t be used. > > The Netbeans folks solved the problem by having their visual installer > install the missing parts after the user accepts the license agreement. Other > projects are going the way of providing a detailed setup instructions on how > the user can manually install the JavaFX parts and then run their software. > > I just wanted to point this out as after we noticed JavaFX is a problem are > currently working on ways to replace it and with my board hat on, I’m > currently having a look at how our projects are handling this (Which is > unfortunately a lot of work) > > Chris > > > > From: Jialin Qiao > Date: Monday, 7. November 2022 at 03:00 > To: dev@iotdb.apache.org > Subject: [VOTE] Create a repo for TsFile-Viewer > Hi, > > Here is a Tool for viewing TsFile: https://github.com/apache/iotdb/pull/6995 > > As TsFile-Viewer could be used independently, it could be a separate > repository for users. > > So, I'd like to start a vote for creating a new Repo for TsFile-Viewer. > > Please vote accordingly: > > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove with the reason > > The vote is open for the next 72 hours and passes if at least three +1 > votes and more +1 votes than -1 votes. > > Thanks, > — > Jialin Qiao > Apache IoTDB PMC
[VOTE] Create a repo for TsFile-Viewer
Hi, Here is a Tool for viewing TsFile: https://github.com/apache/iotdb/pull/6995 As TsFile-Viewer could be used independently, it could be a separate repository for users. So, I'd like to start a vote for creating a new Repo for TsFile-Viewer. Please vote accordingly: [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove with the reason The vote is open for the next 72 hours and passes if at least three +1 votes and more +1 votes than -1 votes. Thanks, — Jialin Qiao Apache IoTDB PMC
Renaming partition region to config node
Hi, Currently, we have some parameters like partition_region_consensus_protocol_class, which actually specify the consensus class of the config node. Therefore, defining an abstract partition region is unnecessary, we'd better rename it as config_node_consensus_protocol_class directly for better comprehension. Besides, I adjust some parameters' locations for convenience. https://github.com/apache/iotdb/pull/7909 Thanks, — Jialin Qiao Apache IoTDB PMC
