AW: [DISCUSS] Apache TsFile Proposal
Ok … so after pinging trademarks on the state of the name-search, they told me
they were still missing something. Hopefully after adding that, they’ll be able
to start their work.
Chris
Von: Jialin Qiao
Datum: Freitag, 27. Oktober 2023 um 13:44
An: dev@iotdb.apache.org
Betreff: Re: [DISCUSS] Apache TsFile Proposal
Hi, Looks great! +1 for name search
―
Jialin Qiao
Apache IoTDB PMC
Christofer Dutz 于2023年10月26日周四 17:19写道：
>
> FYI,
>
> https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-214
>
> Chris
>
> Von: Xiangdong Huang
> Datum: Donnerstag, 26. Oktober 2023 um 04:23
> An: dev@iotdb.apache.org
> Betreff: Re: [DISCUSS] Apache TsFile Proposal
> Hi,
>
> The content looks fine. +1 for name-search.
> Best,
> ---
> Xiangdong Huang
>
>
>
> Christofer Dutz 于2023年10月25日周三 22:24写道：
>
> >
> > More feedback from the board was that the description is pretty long and
> > having it shorted would leave us more wiggle-room in the future.
> > Also, should we formally have passed a name-search … if you’re ok with
> > this, I would start the Name-Search.
> >
> > So that would result in the following resolution:
> >
> > WHEREAS, the Board of Directors deems it to be in the best interests
> > of
> >the Foundation and consistent with the Foundation's purpose to
> >establish a Project Management Committee charged with the creation
> > and
> >maintenance of open-source software, for distribution at no charge
> > to
> >the public, related to a columnar storage file format designed for
> > time
> >series data.
> >
> >NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee
> >(PMC), to be known as the "Apache TsFile Project", be and hereby is
> >established pursuant to Bylaws of the Foundation; and be it further
> >
> >RESOLVED, that the Apache TsFile be and hereby is responsible for
> >the creation and maintenance of software related to a columnar
> > storage
> >file format designed for time series data.
> >
> >RESOLVED, that the office of "Vice President, Apache TsFile" be and
> >hereby is created, the person holding such office to serve at the
> >direction of the Board of Directors as the chair of the Apache
> >TsFile Project, and to have primary responsibility for management of
> >the projects within the scope of responsibility of the Apache TsFile
> >Project; and be it further
> >
> >RESOLVED, that the persons listed immediately below be and hereby
> > are
> >appointed to serve as the initial members of the Apache TsFile
> >Project:
> >
> >- Yuan Tian (jackietien at apache dot org)
> >- Chao Wang (wangchao316 at apache dot org)
> >- Christofer Dutz (cdutz at apache dot org)
> >- Jinrui Zhang (xingtanzjr at apache dot org)
> >- Steve Yurong Su (rong at apache dot org)
> >- Xinyu Tan (tanxinyu at apache dot org)
> >- Haohan Hou (haonan at apache dot org)
> >- Gaofei Cao (gaogaofei at apache dot org)
> >- Jialin Qiao (qiaojialin at apache dot org)
> >- Kun Liu (liukun at apache dot org)
> >- Houliang Qi (neuyilan at apache dot org)
> >- Xiangdong Huang (hxd at apache dot org)
> >- Chao Wang (chaow at apache dot org)
> >- Jianmin Wang (jimwang at apache dot org)
> >- Tian Jiang (jiangtian at apache dot org）
> >- Xinyi Zhao (zhaoxinyi at apache dot org)
> >- Shuo Zhang (shuozhagn at apache dot org)
> >
> >NOW, THEREFORE, BE IT FURTHER RESOLVED, that {name of the chair} be
> >appointed to the office of Vice President, Apache TsFile, to serve
> > in
> >accordance with and subject to the direction of the Board of
> > Directors
> >and the Bylaws of the Foundation until death, resignation,
> > retirement,
> >removal or disqualification, or until a successor is appointed; and
> > be
> >it further
> >
> >RESOLVED, that the Apache TsFile Project be and hereby is tasked
> > with
> >the migration and rationalization of the Apache IoTDB TsFile
> > subproject;
> >and be it further
> >
> >RESOLVED, that all responsibilities pertaining to the Apache IoTDB
> >TsFile subproject encumbered upon the Apache IoTDB PMC are hereafter
> >
AW: [DISCUSS] Apache TsFile Proposal
FYI,
https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-214
Chris
Von: Xiangdong Huang
Datum: Donnerstag, 26. Oktober 2023 um 04:23
An: dev@iotdb.apache.org
Betreff: Re: [DISCUSS] Apache TsFile Proposal
Hi,
The content looks fine. +1 for name-search.
Best,
---
Xiangdong Huang
Christofer Dutz 于2023年10月25日周三 22:24写道：
>
> More feedback from the board was that the description is pretty long and
> having it shorted would leave us more wiggle-room in the future.
> Also, should we formally have passed a name-search … if you’re ok with
> this, I would start the Name-Search.
>
> So that would result in the following resolution:
>
> WHEREAS, the Board of Directors deems it to be in the best interests
> of
>the Foundation and consistent with the Foundation's purpose to
>establish a Project Management Committee charged with the creation
> and
>maintenance of open-source software, for distribution at no charge
> to
>the public, related to a columnar storage file format designed for
> time
>series data.
>
>NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee
>(PMC), to be known as the "Apache TsFile Project", be and hereby is
>established pursuant to Bylaws of the Foundation; and be it further
>
>RESOLVED, that the Apache TsFile be and hereby is responsible for
>the creation and maintenance of software related to a columnar
> storage
>file format designed for time series data.
>
>RESOLVED, that the office of "Vice President, Apache TsFile" be and
>hereby is created, the person holding such office to serve at the
>direction of the Board of Directors as the chair of the Apache
>TsFile Project, and to have primary responsibility for management of
>the projects within the scope of responsibility of the Apache TsFile
>Project; and be it further
>
>RESOLVED, that the persons listed immediately below be and hereby
> are
>appointed to serve as the initial members of the Apache TsFile
>Project:
>
>- Yuan Tian (jackietien at apache dot org)
>- Chao Wang (wangchao316 at apache dot org)
>- Christofer Dutz (cdutz at apache dot org)
>- Jinrui Zhang (xingtanzjr at apache dot org)
>- Steve Yurong Su (rong at apache dot org)
>- Xinyu Tan (tanxinyu at apache dot org)
>- Haohan Hou (haonan at apache dot org)
>- Gaofei Cao (gaogaofei at apache dot org)
>- Jialin Qiao (qiaojialin at apache dot org)
>- Kun Liu (liukun at apache dot org)
>- Houliang Qi (neuyilan at apache dot org)
>- Xiangdong Huang (hxd at apache dot org)
>- Chao Wang (chaow at apache dot org)
>- Jianmin Wang (jimwang at apache dot org)
>- Tian Jiang (jiangtian at apache dot org）
>- Xinyi Zhao (zhaoxinyi at apache dot org)
>- Shuo Zhang (shuozhagn at apache dot org)
>
>NOW, THEREFORE, BE IT FURTHER RESOLVED, that {name of the chair} be
>appointed to the office of Vice President, Apache TsFile, to serve
> in
>accordance with and subject to the direction of the Board of
> Directors
>and the Bylaws of the Foundation until death, resignation,
> retirement,
>removal or disqualification, or until a successor is appointed; and
> be
>it further
>
>RESOLVED, that the Apache TsFile Project be and hereby is tasked
> with
>the migration and rationalization of the Apache IoTDB TsFile
> subproject;
>and be it further
>
>RESOLVED, that all responsibilities pertaining to the Apache IoTDB
> TsFile subproject encumbered upon the Apache IoTDB PMC are hereafter
>discharged.
>
>
>
>
>
> Von: Christofer Dutz
> Datum: Mittwoch, 25. Oktober 2023 um 15:03
> An: dev@iotdb.apache.org
> Betreff: AW: [DISCUSS] Apache TsFile Proposal
> So, it seems that this document is based on an incubation proposal … sorry
> … I just reviewed it on a content-level.
> As we’re going straight TLP this should look quite a bit different. So
> I’ll try to whip up a version:
>
> WHEREAS, the Board of Directors deems it to be in the best interests
> of
>the Foundation and consistent with the Foundation's purpose to
>establish a Project Management Committee charged with the creation
> and
>maintenance of open-source software, for distribution at no charge
> to
>the public, related to a columnar storage file format designed for
> time
>series data, which supports efficient compression, high throughput
&g
AW: [DISCUSS] Apache TsFile Proposal
More feedback from the board was that the description is pretty long and having
it shorted would leave us more wiggle-room in the future.
Also, should we formally have passed a name-search … if you’re ok with this, I
would start the Name-Search.
So that would result in the following resolution:
WHEREAS, the Board of Directors deems it to be in the best interests of
the Foundation and consistent with the Foundation's purpose to
establish a Project Management Committee charged with the creation and
maintenance of open-source software, for distribution at no charge to
the public, related to a columnar storage file format designed for time
series data.
NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee
(PMC), to be known as the "Apache TsFile Project", be and hereby is
established pursuant to Bylaws of the Foundation; and be it further
RESOLVED, that the Apache TsFile be and hereby is responsible for
the creation and maintenance of software related to a columnar storage
file format designed for time series data.
RESOLVED, that the office of "Vice President, Apache TsFile" be and
hereby is created, the person holding such office to serve at the
direction of the Board of Directors as the chair of the Apache
TsFile Project, and to have primary responsibility for management of
the projects within the scope of responsibility of the Apache TsFile
Project; and be it further
RESOLVED, that the persons listed immediately below be and hereby are
appointed to serve as the initial members of the Apache TsFile
Project:
- Yuan Tian (jackietien at apache dot org)
- Chao Wang (wangchao316 at apache dot org)
- Christofer Dutz (cdutz at apache dot org)
- Jinrui Zhang (xingtanzjr at apache dot org)
- Steve Yurong Su (rong at apache dot org)
- Xinyu Tan (tanxinyu at apache dot org)
- Haohan Hou (haonan at apache dot org)
- Gaofei Cao (gaogaofei at apache dot org)
- Jialin Qiao (qiaojialin at apache dot org)
- Kun Liu (liukun at apache dot org)
- Houliang Qi (neuyilan at apache dot org)
- Xiangdong Huang (hxd at apache dot org)
- Chao Wang (chaow at apache dot org)
- Jianmin Wang (jimwang at apache dot org)
- Tian Jiang (jiangtian at apache dot org）
- Xinyi Zhao (zhaoxinyi at apache dot org)
- Shuo Zhang (shuozhagn at apache dot org)
NOW, THEREFORE, BE IT FURTHER RESOLVED, that {name of the chair} be
appointed to the office of Vice President, Apache TsFile, to serve in
accordance with and subject to the direction of the Board of Directors
and the Bylaws of the Foundation until death, resignation, retirement,
removal or disqualification, or until a successor is appointed; and be
it further
RESOLVED, that the Apache TsFile Project be and hereby is tasked with
the migration and rationalization of the Apache IoTDB TsFile subproject;
and be it further
RESOLVED, that all responsibilities pertaining to the Apache IoTDB
TsFile subproject encumbered upon the Apache IoTDB PMC are hereafter
discharged.
Von: Christofer Dutz
Datum: Mittwoch, 25. Oktober 2023 um 15:03
An: dev@iotdb.apache.org
Betreff: AW: [DISCUSS] Apache TsFile Proposal
So, it seems that this document is based on an incubation proposal … sorry … I
just reviewed it on a content-level.
As we’re going straight TLP this should look quite a bit different. So I’ll try
to whip up a version:
WHEREAS, the Board of Directors deems it to be in the best interests of
the Foundation and consistent with the Foundation's purpose to
establish a Project Management Committee charged with the creation and
maintenance of open-source software, for distribution at no charge to
the public, related to a columnar storage file format designed for time
series data, which supports efficient compression, high throughput of
read and write, and compatibility with various frameworks, such as Spark
and Flink. It is easy to integrate TsFile into IoT big data processing
frameworks.
NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee
(PMC), to be known as the "Apache TsFile Project", be and hereby is
established pursuant to Bylaws of the Foundation; and be it further
RESOLVED, that the Apache TsFile be and hereby is responsible for
the creation and maintenance of software related to a columnar storage
file format designed for time series data, which supports efficient
compression, high throughput of read and write, and compatibility with
various frameworks, such as Spark and Flink. It is easy to integrate
TsFile
into IoT big data processing frameworks.
AW: [DISCUSS] Apache TsFile Proposal
So, it seems that this document is based on an incubation proposal … sorry … I
just reviewed it on a content-level.
As we’re going straight TLP this should look quite a bit different. So I’ll try
to whip up a version:
WHEREAS, the Board of Directors deems it to be in the best interests of
the Foundation and consistent with the Foundation's purpose to
establish a Project Management Committee charged with the creation and
maintenance of open-source software, for distribution at no charge to
the public, related to a columnar storage file format designed for time
series data, which supports efficient compression, high throughput of
read and write, and compatibility with various frameworks, such as Spark
and Flink. It is easy to integrate TsFile into IoT big data processing
frameworks.
NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee
(PMC), to be known as the "Apache TsFile Project", be and hereby is
established pursuant to Bylaws of the Foundation; and be it further
RESOLVED, that the Apache TsFile be and hereby is responsible for
the creation and maintenance of software related to a columnar storage
file format designed for time series data, which supports efficient
compression, high throughput of read and write, and compatibility with
various frameworks, such as Spark and Flink. It is easy to integrate
TsFile
into IoT big data processing frameworks.
RESOLVED, that the office of "Vice President, Apache TsFile" be and
hereby is created, the person holding such office to serve at the
direction of the Board of Directors as the chair of the Apache
TsFile Project, and to have primary responsibility for management of
the projects within the scope of responsibility of the Apache TsFile
Project; and be it further
RESOLVED, that the persons listed immediately below be and hereby are
appointed to serve as the initial members of the Apache TsFile
Project:
- Yuan Tian (jackietien at apache dot org)
- Chao Wang (wangchao316 at apache dot org)
- Christofer Dutz (cdutz at apache dot org)
- Jinrui Zhang (xingtanzjr at apache dot org)
- Steve Yurong Su (rong at apache dot org)
- Xinyu Tan (tanxinyu at apache dot org)
- Haohan Hou (haonan at apache dot org)
- Gaofei Cao (gaogaofei at apache dot org)
- Jialin Qiao (qiaojialin at apache dot org)
- Kun Liu (liukun at apache dot org)
- Houliang Qi (neuyilan at apache dot org)
- Xiangdong Huang (hxd at apache dot org)
- Chao Wang (chaow at apache dot org)
- Jianmin Wang (jimwang at apache dot org)
- Tian Jiang (jiangtian at apache dot org）
- Xinyi Zhao (zhaoxinyi at apache dot org)
- Shuo Zhang (shuozhagn at apache dot org)
NOW, THEREFORE, BE IT FURTHER RESOLVED, that {name of the chair} be
appointed to the office of Vice President, Apache TsFile, to serve in
accordance with and subject to the direction of the Board of Directors
and the Bylaws of the Foundation until death, resignation, retirement,
removal or disqualification, or until a successor is appointed; and be
it further
RESOLVED, that the Apache TsFile Project be and hereby is tasked with
the migration and rationalization of the Apache IoTDB TsFile subproject;
and be it further
RESOLVED, that all responsibilities pertaining to the Apache IoTDB
TsFile subproject encumbered upon the Apache IoTDB PMC are hereafter
discharged.
Von: Christofer Dutz
Datum: Mittwoch, 25. Oktober 2023 um 12:24
An: dev@iotdb.apache.org
Betreff: AW: [DISCUSS] Apache TsFile Proposal
So, I gave the text a little run and tweaked a few parts (I do hope I based
that on the latest version discussed here):
# Abstract
TsFile is a columnar storage file format designed for time series data, which
supports efficient compression, high throughput of read and write, and
compatibility with various frameworks, such as Spark and Flink. It is easy to
integrate TsFile into IoT big data processing frameworks.
# Proposal
TsFile is used for managing time series data. Although it is firstly used
inside IoTDB, many users and companies use TsFile directly as a time series
data management solution independently. Besides, there is a growing requirement
for Multi-Language Support of TsFile implementations, such as C++, Go and Rust.
Apache IoTDB community hereby submit this proposal of TsFile as an independent
Apache project. This proposal outlines the key features and benefits of TsFile,
along with the integration plan and the need for multi-language support.
# Background
Time series data is becoming increasingly important in a wide range of
applications, including IoT, intelligent control, finance, log analysis, and
AW: [DISCUSS] Apache TsFile Proposal
So, I gave the text a little run and tweaked a few parts (I do hope I based that on the latest version discussed here): # Abstract TsFile is a columnar storage file format designed for time series data, which supports efficient compression, high throughput of read and write, and compatibility with various frameworks, such as Spark and Flink. It is easy to integrate TsFile into IoT big data processing frameworks. # Proposal TsFile is used for managing time series data. Although it is firstly used inside IoTDB, many users and companies use TsFile directly as a time series data management solution independently. Besides, there is a growing requirement for Multi-Language Support of TsFile implementations, such as C++, Go and Rust. Apache IoTDB community hereby submit this proposal of TsFile as an independent Apache project. This proposal outlines the key features and benefits of TsFile, along with the integration plan and the need for multi-language support. # Background Time series data is becoming increasingly important in a wide range of applications, including IoT, intelligent control, finance, log analysis, and monitoring systems. TsFile has been developed by the Apache IoTDB community in Java and is currently located inside the Apache IoTDB code repository. Users could store time series data using TsFile, then read and analyze it in IoTDB, Spark and Flink. IoTDB could also generate TsFiles and synchronize TsFiles between two IoTDB instances. Furthermore, the demand for TsFile implementations in multiple programming languages has been growing, as it allows developers to leverage TsFile's capabilities in their preferred language. # TsFile offers several distinctive features and benefits: Efficient Storage and Compression: TsFile employs advanced compression techniques to minimize storage requirements, resulting in reduced disk space consumption and improved system efficiency. Flexible Schema and Metadata Management: TsFile allows for directly write data without predefining the schema, which is flexible for data acquisition. High Query Performance with time range: TsFile has indexed devices, sensors, and time dimensions to accelerate query performance, enabling fast filtering and retrieval of time series data. Seamless Integration: TsFile is designed to seamlessly integrate with existing big data frameworks, such as Spark, Flink and Hadoop. # Rationale TsFile is the first existing standard file format for time series data. In the industry companies usually write time series data without unification or use general columnar file format such as Parquet and ORC, which makes data collection and processing complicated without a standard. With TsFile, organizations could write data in TsFile inside end devices or gateway, then transfer TsFile to the cloud for unified management in IoTDB and other systems. In this way, we lower the network transmission and the computing resource consumption in the cloud. # Initial Goals The initial goals include: Make TsFile an independent project that has its own SDK and documentation that is easier to use. Multi-Language Support of TsFile implementations apart from Java, such as C++, Go and Rust. Integrate more encoding and compression methods in TsFile. More tools for TsFile: visualization tool, parsing tool, repair tool. # Current Status ## Meritocracy We plan to invite a subset of the Apache IoTDB committers as initial committers of Apache TsFile. And we would like to follow ASF meritocratic principles and invite additional developers to participate. We will establish the documentation and encourage and monitor community participation so that privileges can be extended to those that contribute. ## Community The TsFile community is grown from the Apache IoTDB Community. The IoTDB community is introducing TsFile at many technical conferences. Next, we will build the mailing list for more convenience, broader communication and archived discussions. We are open to recruiting more developers from diverse backgrounds. ## Core Developers TsFile initial PMCs are from IoTDB community: Christofer Dutz, Xiangdong Huang, Jialin Qiao, Steve Yurong Su, Jinrui Zhang, Yuan Tian, Xinyu Tan, Haonan Hou, Gaofei Cao, Tian Jiang, Chao Wang(wangchao316), Chao Wang(mychaow), Houliang Qi, Kun Liu. These people have extensive experience in building database and data management system. ## Alignment The ASF is the natural choice to host the TsFile project as its goal of encouraging community-driven open-source projects fits with our vision for TsFile. Additionally, many other projects with which we are familiar with and expect TsFile to integrate with, such as Apache Spark, Apache Flink and Apache IoTDB are hosted by the ASF and we will benefit and provide benefits in close proximity to them. # Known Risks ## Project Name TsFile project is used in IoTDB and other scenarios for over 7 years, its name is unique. ## Orphaned Products The
