Introduction to Apache IoTDB C# Client

[Xiulong Yuan]

Hi guys, I am Yuan Xiulong from China. I'm currently pursuing my master's
degree in School of Software, Tsinghua University. In April last year, my
mentor, Professor Liu Lin, suggested that I, Lu Zhan and Zeng Zheng should
develop a C# client for Apache IoTDB so that C# users can easily use Apache
IoTDB. After weeks of research and development, we open-sourced the client
and released it on NuGet
 in May of last
year[1]. Since its release, our C# client has received thousands of
downloads and also received a lot of feedback and improvement suggestions
from our community: such as providing a more user-friendly interface,
providing parallel request sending and processing
,
and supporting session abnormal reconnecting
[2]
(support from Lu Zhan), etc. At the same time, our C# client is also timely
updated to support the latest features of Apache IoTDB, such as aligned
time series insertion, SchemaTemplate manipulation interface support, and
Tablet structure that supports inserting null values
[3]
(support from Lu Zhan), etc. We even had a developer from Lithuania
involved in refactoring the code structure for us. In this blog, we will
give a basic introduction to the key features of our C# client, the
installation guide and examples.
Installation
For easier installation, we have prepared NuGet packages for C# users. You
can directly install the client through .NET CLI. The NuGet package is
linked as follows  [4]. Run
the following command on your command line to complete the installation of
the latest version of the client. At the same time, in order to support
better cross-platform, our client is developed and constructed based on
.NET SDK (supported by Zeng Zheng).

dotnet add package Apache.IoTDB

Features
SessionPool
Similar to the Java client of Apache IoTDB, we also provide an
implementation of SessionPool that supports parallel request sending and
processing. In scenarios where network requests take a long time or are
unstable, SessionPool can provide better performance than a single client.
Regarding SessionPool, we provide detailed implementation details and
performance tests in the document. For details, see the document link

[5] (support from Ma Wenxuan).
ByteBuffer
Data communication between the client and the server requires a lot of
serialization and deserialization operations according to the Apache IoTDB
serialization protocol. Simply using native C# serialization method will
bring severe memory allocation and free overhead. We provide ByteBuffer
which can estimateof the memory size of large tablets and use a smart
algorithm to expand the memory size to improve the performance of
serialization and deserialization during serializing large tablet.
Serialization and deserialization using the implemented ByteBuffer can
improve the performance by 30 times compared with C# native serialization.
See document

[6] for details.
Example
In this section, we provide code examples for the use of the C# client. All
interfaces exposed by the C# client are asynchronous interfaces. You start
with establishing a sessionpool by specifying the IP, port and size of the
pool. The size of the SessionPool represents the number of local
connections established with the server.

var session_pool = new SessionPool(host, port, pool_size);
await session_pool.Open(false);

After setting up the SessionPool, we can use it to create a timeseries:

status = await session_pool.CreateTimeSeries(

"root.97209_TEST_CSHARP_CLIENT_GROUP.TEST_CSHARP_CLIENT_DEVICE.TEST_CSHARP_CLIENT_TS1",
TSDataType.TEXT,
TSEncoding.PLAIN, Compressor.UNCOMPRESSED);

We can use SessionPool for row-by-row Record insertion. Since SessionPool
can support parallel sending, when sending a large number of Records,
SessionPool will provide better performance than a single client
connection.

var tasks = new List>();
for (var timestamp = 1; timestamp <= fetch_size * processed_size; timestamp++)
{
var rowRecord = new RowRecord(timestamp, values, measures);
var task = session_pool.InsertRecordAsync(
"root.97209_TEST_CSHARP_CLIENT_GROUP.TEST_CSHARP_CLIENT_DEVICE",
rowRecord);
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());

We can also use SessionPool to execute SQL statements and display the
results.

var res = await session_pool.ExecuteQueryStatementAsync(
"select * from " + string.Format("{0}.{1}",
test_group_name, 

?????? ????????????

[????????????]

Thank you to the community. I will try my best to complete the work of the 
community and do something meaningful


zhengqiang


CISDI Info






----
??: "Xiangdong Huang"

Re: 账户权限申请

[Xiangdong Huang]

Hi weihao,

Welcome, I have add you to the contributor group on JIRA and give the write
permission to Confluence.
Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


李** <18110526...@163.com> 于2022年5月12日周四 10:24写道：

> self-introduction：I'm WeiHao Li, a development engineer of IoTDB team of
> Timecho.
>
>
> Jira ID：Weihao Li
>
>
> Confluence ID：Weihao Li

账户权限申请

[李**]

self-introduction：I'm WeiHao Li, a development engineer of IoTDB team of 
Timecho.


Jira ID：Weihao Li


Confluence ID：Weihao Li

Re: [Discuss] New all-in-one distribution package structure

[Jialin Qiao]

Hi,

I previously expect the all-in-one contain a server and a CLI, which we
used for deploying IoTDB（cluster or standalone） and doing some quick
operations.

But now, I thought CLI could just be a part of IoTDB-cluster, like a csv
tool or so.

So, I prefer to make all-in-one a truly all-in-one:

IoTDB-cluster or IoTDB: cluster( with standalone ) + cli

All-in-one: cluster + library-udf + grafana-plugin + rocksdb(optional).

Thanks,
—
Jialin Qiao
Apache IoTDB PMC


Xiangdong Huang  于2022年5月11日周三 16:01写道：

> Hi,
>
> Before discussing the new release, we need to know why there is
> "all-in-one" and define what is "all-in-one" and what is not.
>
> We released "all-in-one" at that time because we have a standalone version,
> the cluster version, and grafana-connector.
> And, we thought:
> - users may just run IoTDB on one node as a service, so the standalone
> version is fine.
> - users may need to run IoTDB on multiple nodes as a service, so the
> cluster version is suitable;
> - grafana-connector is an additional service of IoTDB.
>
>
> Now, we have:
> 1. the standalone version, which only uses datanode module.
> 2. the cluster version, which uses datanode and confignode modules.
> 3. the replaceable components: rocksdb-metadata
> 4. the pluggable library: library-udf
> 5. the client libraries: jdbc, java, c++, etc..
> 6. the grafana connector (deprecated?)
> 7. the grafana plugin
> 8. the iotdb-web-workbench (I think it is not ready for an apache release)
>
> So, which modules that "all-in-one" wants to contain?
>
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
>
>  黄向东
> 清华大学 软件学院
>
>
> Haonan Hou  于2022年5月11日周三 14:46写道：
>
> > Hi,
> >
> > I submit a PR[1] about adding config node to distribution package. There
> > is a change of the structure of all-in-one
> > package structure. Please refer the PR link below for details.
> >
> > Please let me know if you have any suggestion on this. Thanks!
> >
> > [1] https://github.com/apache/iotdb/pull/5861
> >
> > BR,
> > Haonan Hou
> >
> >
> >
>

[BUILD-FAILURE]: Job 'IoTDB/IoTDB-Pipe/master [master] [588]'

[Apache Jenkins Server]

BUILD-FAILURE: Job 'IoTDB/IoTDB-Pipe/master [master] [588]':

Check console output at "https://ci-builds.apache.org/job/IoTDB/job/IoTDB-Pipe/job/master/588/;>IoTDB/IoTDB-Pipe/master
 [master] [588]"

回复: [Discuss] Remove old cluster module from distribution package

[刘 旭鑫]

+1

发件人: HW-Chao Wang <576749...@qq.com.INVALID>
发送时间: 2022年5月11日 6:01
收件人: dev 
主题: Re: [Discuss] Remove old cluster module from distribution package

+1



---Original---
From: "Jialin Qiao"

Re: [Discuss] New all-in-one distribution package structure

[Xiangdong Huang]

Hi,

Before discussing the new release, we need to know why there is
"all-in-one" and define what is "all-in-one" and what is not.

We released "all-in-one" at that time because we have a standalone version,
the cluster version, and grafana-connector.
And, we thought:
- users may just run IoTDB on one node as a service, so the standalone
version is fine.
- users may need to run IoTDB on multiple nodes as a service, so the
cluster version is suitable;
- grafana-connector is an additional service of IoTDB.


Now, we have:
1. the standalone version, which only uses datanode module.
2. the cluster version, which uses datanode and confignode modules.
3. the replaceable components: rocksdb-metadata
4. the pluggable library: library-udf
5. the client libraries: jdbc, java, c++, etc..
6. the grafana connector (deprecated?)
7. the grafana plugin
8. the iotdb-web-workbench (I think it is not ready for an apache release)

So, which modules that "all-in-one" wants to contain?

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Haonan Hou  于2022年5月11日周三 14:46写道：

> Hi,
>
> I submit a PR[1] about adding config node to distribution package. There
> is a change of the structure of all-in-one
> package structure. Please refer the PR link below for details.
>
> Please let me know if you have any suggestion on this. Thanks!
>
> [1] https://github.com/apache/iotdb/pull/5861
>
> BR,
> Haonan Hou
>
>
>

[Discuss] New all-in-one distribution package structure

[Haonan Hou]

Hi, 

I submit a PR[1] about adding config node to distribution package. There is a 
change of the structure of all-in-one
package structure. Please refer the PR link below for details.

Please let me know if you have any suggestion on this. Thanks!

[1] https://github.com/apache/iotdb/pull/5861

BR,
Haonan Hou

Re: [Discuss] Remove old cluster module from distribution package

[HW-Chao Wang]

+1



---Original---
From: "Jialin Qiao"