[
https://issues.apache.org/jira/browse/GRIFFIN-332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173586#comment-17173586
]
ishan verma commented on GRIFFIN-332:
-------------------------------------
Thanks [~obaid].
yes same thing i have tried with type=jdbc , but i want to create measure using
mysql data sources and show the results in metric as i have done using HIVE.
But while creating measure its showing error as :
*"type jdbc is not present in[HIVE,CUSTOM,KAFKA,AVRO]"*
Have you tried to create measure using this jdbc config? I see that you are not
using UI part.
Also i had a discussion with [~wankun] on this, he said that :-
"the UI module has not update to support this measure definition. As
workaround ,I had push a new request to import a shell tool, which use the
measure module directly to support this task.
Also I with someone could update the UI as soon as possible."
So i think its not possible right now to create measure using mysql source,
maybe in future.
Also i see that you are storing output to S3. could you give me a little
overview on this, where you have specified your config for s3 in code, how its
fetching output.
once again thanks for your clarification :)
> JDBC Connector: Ability to Select Specific Columns Instead of All the Columns
> -----------------------------------------------------------------------------
>
> Key: GRIFFIN-332
> URL: https://issues.apache.org/jira/browse/GRIFFIN-332
> Project: Griffin
> Issue Type: Improvement
> Components: accuracy-batch
> Affects Versions: 0.6.0
> Reporter: Obaidul Karim
> Priority: Major
> Labels: columns, jdbc
>
> *Background:*
> Thanks to https://issues.apache.org/jira/browse/GRIFFIN-315, we already have
> JDBC connector.
> However, currently, it is pulling all the columns using`"SELECT * FROM
> $fullTableName"`.
> It will cause some issues for larger JDBC tables -
> - memory overhead for spark data frame
> - longer execution time
> - resource overhear for RDBMS
> *Proposed Improvement:*
> So, I propose the feature to allow JDBC connector to able to select only
> required columns.
> *Example:*
> We have a rule `"rule":"src.id = tgt.id and src.country = tgt.country "`.
> Then we only need two columns `id` and 'country'.
> So, in connector we can add additional clause `columns` to select only
> required columns, like below:
>
> {code:java}
> { "name":"src",
> "connector":{ "type":"jdbc",
> "config":{ "database":"mydatabase",
> "tablename":"mytable",
> "columns":"id, country",
> "url":"jdbc:sqlserver://myhost:1433;databaseName=mydatabase",
> "user":"user",
> "password":"password",
> "driver":"com.microsoft.sqlserver.jdbc.SQLServerDriver",
> "where":""
> }
> }
> }
> {code}
> We can implement it like this, if there is `columns` clause then use it
> otherwise use `*` as default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
