[jira] [Commented] (GRIFFIN-173) [Measure] Support JDBC connection as data source
[ https://issues.apache.org/jira/browse/GRIFFIN-173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534369#comment-16534369 ] Lionel Liu commented on GRIFFIN-173: [~maver1ck], spark streaming splits data from streaming source like kafka into mini-batches, and generate data sets with the arriving timestamps for each, thus the DQ metric in streaming mode actually indicates the metrics of these mini-batches. The TimestampStorage in streaming data connector stores the timestamps, while in batch data connector, to keep the consistency, we store the application timestamp as well. The timestamps stored helps in streaming process, not important for batch process. > [Measure] Support JDBC connection as data source > > > Key: GRIFFIN-173 > URL: https://issues.apache.org/jira/browse/GRIFFIN-173 > Project: Griffin (Incubating) > Issue Type: Task >Reporter: Maciej Bryński >Assignee: William Guo >Priority: Major > Original Estimate: 72h > Remaining Estimate: 72h > > DoD: > Support JDBC connection as data source. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (GRIFFIN-173) [Measure] Support JDBC connection as data source
[ https://issues.apache.org/jira/browse/GRIFFIN-173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1652#comment-1652 ] Lionel Liu commented on GRIFFIN-173: Hi [~maver1ck], I agree with you. Spark supports JDBC already, which makes it much easier. The only thing we need to do is to implement a new batch data connector for JDBC, with "JDBC" type supported here: [https://github.com/apache/incubator-griffin/blob/master/measure/src/main/scala/org/apache/griffin/measure/datasource/connector/DataConnectorFactory.scala#L59|https://github.com/apache/incubator-griffin/blob/master/measure/src/main/scala/org/apache/griffin/measure/datasource/connector/DataConnectorFactory.scala#L59),] I suggest you have a try and test for this, that would be a good chance to know Griffin and help it grow. Thanks, Lionel > [Measure] Support JDBC connection as data source > > > Key: GRIFFIN-173 > URL: https://issues.apache.org/jira/browse/GRIFFIN-173 > Project: Griffin (Incubating) > Issue Type: Task >Reporter: Maciej Bryński >Assignee: William Guo >Priority: Major > Original Estimate: 72h > Remaining Estimate: 72h > > DoD: > Support JDBC connection as data source. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (GRIFFIN-173) [Measure] Support JDBC connection as data source
[ https://issues.apache.org/jira/browse/GRIFFIN-173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16533135#comment-16533135 ] William Guo commented on GRIFFIN-173: - hi Maciej Bryński , Let's figure out how to make it, though, recently, we might not have priority on this feature. Do you want to discuss and contribute for this feature? Thanks, William > [Measure] Support JDBC connection as data source > > > Key: GRIFFIN-173 > URL: https://issues.apache.org/jira/browse/GRIFFIN-173 > Project: Griffin (Incubating) > Issue Type: Task >Reporter: Maciej Bryński >Assignee: William Guo >Priority: Major > Original Estimate: 72h > Remaining Estimate: 72h > > DoD: > Support JDBC connection as data source. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
