Ravindra Pesala created CARBONDATA-1522:
-------------------------------------------
Summary: 6. Loading aggregation tables for streaming data tables.
Key: CARBONDATA-1522
URL: https://issues.apache.org/jira/browse/CARBONDATA-1522
Project: CarbonData
Issue Type: Sub-task
Reporter: Ravindra Pesala
we can finish the segment load after receives configurable amount of data and
create a new segment to load new streaming data. While finishing the segment we
can trigger the agg tables on it. So there would not be any agg tables on
ongoing streaming segment but querying can be done on the streaming segment of
actual table.
For example user configures stream_segment size as 1 GB, so for every 1 GB of
stream data it receives it creates a new segment and finishes the current
segment. While finishing the current segment we can trigger agg table loading
and compaction of segments.
While querying of data we change the query plan to apply union of agg table and
streaming segment of actual table to get the current data.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
