Dear Haonan, Thank you for the quick and encouraging feedback. Your "functionality-first" approach makes perfect sense; I have adjusted my project goals to prioritize stability and accurate mapping over early-stage performance optimization.
Based on our discussion, I have refined the core scope of my proposal as follows: *1. Core Objectives (Primary Focus)* - *Unified Schema Mapping:* Establishing a robust mapping layer between Flink's LogicalType and the IoTDB Relational model (Mapping Time, Tags, and Fields). - *Dynamic Table Sink:* Implementing a DynamicTableSink that ensures "At-Least-Once" delivery semantics using Flink’s checkpointing. - *Dynamic Table Source:* Implementing a DynamicTableSource to allow Flink to scan IoTDB tables, focusing on correct data retrieval and basic column projection. *2. Iterative Optimizations (Secondary Focus)* - *Predicate Pushdown:* Implementing SupportsFilterPushDown to move SQL WHERE clauses (Time/Tags) directly into IoTDB. - *Tablet Batching:* Transitioning to optimized Tablet API writes once the core CRUD operations are stable. *Request for Architectural Pointers* To help me finalize the 350-hour "Implementation Timeline" in my proposal, I would love to take you up on your offer for architectural notes. I am particularly interested in: - *Codebase Pointers:* Which classes in iotdb-core or iotdb-client are the best references for implementing *relational tablet* insertion and querying? - *Schema Metadata:* How does IoTDB currently expose table metadata (identifying which columns are *TAGS* vs. *FIELDS*) to external clients for schema discovery? I am currently working on the manual PoC mapping and will share my progress with the community as soon as it's ready Best regards, Mahesh Vanekar On Thu, 19 Mar 2026 at 13:43, Haonan Hou <[email protected]> wrote: > Dear Mahesh, > > Thank you for your email and for formally introducing yourself as a > candidate for the "Flink connector for IoTDB 2.X Table Mode" project. It's > great to see your enthusiasm and that you've already started reviewing the > iotdb-extras repository, setting up a local environment with IoTDB 2.x + > Flink 1.18, and even planning a PoC. Your initial technical assessment is > very solid! > > Regarding your question about prioritizing specific performance benchmarks > or architectural constraints in the early stages of your proposal: > > *In the initial phase, we recommend not focusing too heavily on > performance optimization yet.* The priority should be to first get the > core functionality working correctly and reliably: > > - Accurate schema mapping from IoTDB Table Mode (Timestamps, Tags, > Fields) to Flink’s LogicalType system > - Proper implementation of DynamicTableSource and DynamicTableSink > - Basic read/write support through Flink SQL and the Table API > > Once the fundamental features are stable and passing basic tests, we can > then iteratively optimize performance (such as Tablet batch writes, filter > pushdown for Tags/time ranges, etc.). This “functionality-first, > optimize-later” approach usually leads to cleaner code and more effective > improvements later on. > > Your planned PoC (manual mapping of Flink RowData to IoTDB Tablet) is an > excellent next step — feel free to share any early results or questions you > encounter. > > If you need any architectural notes on the 2.x Table Mode or pointers to > relevant parts of the codebase while drafting your 350-hour timeline, just > let me know. > > We’re excited about your proposal and look forward to reviewing it. Keep > up the great work! > > Best regards, Haonan Hou > > On Mar 13, 2026, at 11:32 AM, V Mahesh <[email protected]> wrote: > > Dear Haonan Hou and the Apache IoTDB Community, > > I am writing to formally introduce myself as a candidate for the "Flink > connector for IoTDB 2.X Table Mode" project for GSoC 2026. > > As a recent Computer science graduate from CBIT Hyderabad specializing in > IoT and Cybersecurity, I have a strong academic foundation in time-series > data and secure system integration. I am particularly excited about this > project because it addresses a critical bridge in the IoT > ecosystem—enabling real-time stream processing for the modern, > relational-style Table Mode introduced in IoTDB 2.x. > > Initial Technical Assessment > After reviewing the iotdb-extras repository and the Flink 1.18+ > documentation, I have identified the following key focus areas for my > proposal: > 1.) Schema Mapping: Transitioning from the hierarchical Tree Mode to a > structured mapping of Timestamps, Tags (ID columns), and Fields > (Measurement columns) within Flink’s LogicalType system. > 2.) Dynamic Table Stack: Implementing the DynamicTableSink and > DynamicTableSource interfaces to ensure the connector is compatible with > Flink SQL and the Table API. > 3.) Performance: Utilizing the Tablet interface in the IoTDB Java Session > API to ensure high-throughput batch writes, which is essential for > industrial IoT workloads. > 4.) Optimization: Exploring Filter Pushdown (e.g., pushing WHERE clauses > for Tags/Time ranges directly to IoTDB) to minimize data transfer overhead. > > Current Progress > I have already begun setting up a local development environment with IoTDB > 2.x and Flink 1.18 to test basic connectivity. My next step is to create a > small Proof-of-Concept (PoC) demonstrating a manual mapping of a Flink > RowData object to an IoTDB Table Tablet. > > Request for Guidance > As I refine my detailed 350-hour implementation timeline, are there any > specific performance benchmarks or existing architectural constraints > within the 2.x Table Mode that I should prioritize in the early stages of > my proposal? > > I look forward to contributing to the Apache Software Foundation and > working closely with the community. > > Best regards, > Mahesh Vanekar (mail: [email protected]) > https://github.com/Maheshv1204 > > https://www.linkedin.com/in/mahesh-vanekar-18260226b/ > > >
