vagetablechicken edited a comment on issue #3469:
URL:
https://github.com/apache/incubator-doris/issues/3469#issuecomment-639358916
### Prepare Phase Detail Design
#### Sender BE:
```
Get batch from json_scanner
For partialrow in batch:
// If missing key cols, returns error. Otherwise returns ok.
// If we want to support default values(only for non-key cols) for each
write txn,
// those default values need to be pass to receiver BE
// (e.g. PTabletWriterOpenRequest adds a field). And no need to
// modify rows here, coz we only need key cols here. Filling non-key
cols
// could exec in commit phase.
RETURN_IF_ERROR(calc_partition_and_tablet(partialrow, &channel,
&tablet_id))
channel.add_row(partialrow, tablet_id)
```
#### Receiver BE:
```
Get batch from sender BE
For partialrow, tablet_id of batch:
get delta_writer by tablet_id
delta_writer.write(partialrow)
```
##### DeltaWriter(DeltaWriter has two mode, one for disk engine, one for
memory engine. Sets up the mode upon initialization)
```
// change arg Tuple* to void*
// disk engine MemTable will convert it to Tuple
// memory engine MemTable will convert it to PartialRow
DeltaWriter::insert(void* row):
mem_table.insert(row)
if memory_usage > config:
_flush_memtable_async()
_reset_mem_table()
```
##### WTDMemTable(WriteTxnData MemTable, flush WriteTxnData files)
Member var: _buffer, ...
```
WTDMemTable::insert(void* row):
append(_buffer, row, sizeof(uint64_t)+*(uint64_t*)row)
num_rows++;
WTDMemTable::flush():
// footer will save the default values of non-key cols in schema
part(ColumnMetaPB).
// Use the default values for this write txn, otherwise use the default
values
// in tablet schema.
append(_buffer, footer)
write_to_disk(_buffer)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
