[
https://issues.apache.org/jira/browse/GRIFFIN-173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534369#comment-16534369
]
Lionel Liu commented on GRIFFIN-173:
------------------------------------
[~maver1ck], spark streaming splits data from streaming source like kafka into
mini-batches, and generate data sets with the arriving timestamps for each,
thus the DQ metric in streaming mode actually indicates the metrics of these
mini-batches.
The TimestampStorage in streaming data connector stores the timestamps, while
in batch data connector, to keep the consistency, we store the application
timestamp as well.
The timestamps stored helps in streaming process, not important for batch
process.
> [Measure] Support JDBC connection as data source
> ------------------------------------------------
>
> Key: GRIFFIN-173
> URL: https://issues.apache.org/jira/browse/GRIFFIN-173
> Project: Griffin (Incubating)
> Issue Type: Task
> Reporter: Maciej Bryński
> Assignee: William Guo
> Priority: Major
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> DoD:
> Support JDBC connection as data source.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
