gamblewin commented on issue #9093: URL: https://github.com/apache/hudi/issues/9093#issuecomment-1614448235
@danny0405 Thx for replying. 1. Data is committed into the table, but can not be queried by using `sTableEnv.sqlQuery(select * from dept)`.  2. If i use sql way, which is inserting multiple rows in one sql and executing this sql, **is this way bulk insert or not?** ```java sEnv = StreamExecutionEnvironment.getExecutionEnvironment(); sEnv.setRuntimeMode(RuntimeExecutionMode.BATCH); // set execution mode as batch sTableEnv = StreamTableEnvironment.create(sEnv); sEnv.setParallelism(1); sEnv.enableCheckpointing(3000); // SQL way: insert multiple rows in one sql without explicitly configuring write option as bulk insert sTableEnv.executeSql("insert into dept values (1, 'a', NOW()), (2, 'b', NOW())"); ``` 3. If the above sql way is not bulk insert, **is there any way i can bulk insert data by using sql?** I know that for query sql, we can add options to set up some configurations, but i tried add options to insert data sql, it's not working. ```sql insert into dept values (1, 'a', NOW()), (2, 'b', NOW()) /*+ options ( 'write.operation' = 'bulk_insert' )*/ ``` 4. I think what u really mean is using streaming API to bulk insert data. In my understanding, bulk insert means insert a batch of data at a time, but in the following code, **source data is an unbounded stream, how does sink function split source data into different batches?** ```java DataStream<RowData> dataStream = env.addSource(...); Map<String, String> options = new HashMap<>(); // other option configurations ...... options.put("write.operation", "bulk_insert"); DataStream<RowData> dataStream = sEnv.addSource(...); HoodiePipeline.Builder builder = HoodiePipeline.builder("dept") .column(...) .options(options); builder.sink(dataStream, false); ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
