[
https://issues.apache.org/jira/browse/CALCITE-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15531009#comment-15531009
]
slim bouguerra commented on CALCITE-1391:
-----------------------------------------
I think the scope of this request is not very clear. I will try to explain what
druid is capable of hope that help to make the question more narrow.
First let's keep in mind that druid is built around a very strong design
hypothesis which is the immutability of the segments. By having a read only
segments we can achieve huge speedup/caching.
But this it does not mean that you can not insert/append or delete data.
Assuming your are ingestion data via batch ingestion, let's start with the
simple case which is delete data.
Druid has the concept of load and drop rules for segments, hence to drop an
entire segment all we need to do is to send a drop request to the coordinator.
For instance in a reporting use case, usually you want to load the data for the
last 3 months only, you can tell druid to only keep the segment within the last
3 months.
Although if your use case is to delete one or N-rows from one segment that is
not trivial. What you need to do is to send a new index task with the new data
for that specific interval then druid will replace the old segment and by the
new one.
What this imply is that you won't really impact how fast or slow is the query.
But druid will happily answer to the query using the old segment till the
re-index task is done.
For append/insert druid support delta batch ingestion. What his means you can
trigger an index job and you tell druid that you want to append/insert this set
of rows to this existing segment.
What is the impact of this on the query. The answer is the same as above. Druid
will answer to the query with the old data till the index map-reduce job is
done.
If you use case is realtime it is totally a different discussion.
[~arupadhy] hope this make sense to you. Please feel free to explain more the
use case you are targeting.
> CRUD operations using Calcite for DRUID
> ---------------------------------------
>
> Key: CALCITE-1391
> URL: https://issues.apache.org/jira/browse/CALCITE-1391
> Project: Calcite
> Issue Type: Wish
> Components: druid
> Environment: Druid Production Environment
> Reporter: Arvind
> Assignee: Julian Hyde
> Priority: Critical
>
> Hi Team,
> I know that we are able to read data from druid using Calcite SQLLINE command
> but I would like to know if we can insert, Update and delete segments in
> DRUID using Calcite. We are using DRUID as a reporting platform and for us to
> incrementally update, delete and append data we need these operations. As we
> are dealing with huge volumes of data we depend on DRUID and very badly need
> these Insert, Update and Delete.
> Any help is much appreciated.
> Thanks,
> Arvind
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
