[jira] [Commented] (GRIFFIN-338) Custom Connector for mysql not working
[
https://issues.apache.org/jira/browse/GRIFFIN-338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167443#comment-17167443
]
ishan verma commented on GRIFFIN-338:
-
Thanks [~chitralverma] . So what i understood is currently i am using griffin
0.5 and the link you have provided is for the docker image 0.2(i have updated
to 0.5 now) if i am not wrong.
-So should i revert back to 0.2 ?
-or should i use the class given for mysqldataconnector in your provided link
and give it a try using type=CUSTOM ?
[https://github.com/apache/griffin/blob/48f304dc316ed380fef2c5c7891f7ec41e0ddd30/measure/src/main/scala/org/apache/griffin/measure/datasource/connector/batch/MySqlDataConnector.scala]
> Custom Connector for mysql not working
> --
>
> Key: GRIFFIN-338
> URL: https://issues.apache.org/jira/browse/GRIFFIN-338
> Project: Griffin
> Issue Type: Bug
> Environment: griffin version= 0.5, mysql on EC2 instance using
> griffin docker image
>Reporter: ishan verma
>Priority: Critical
> Attachments: image-2020-07-29-20-23-40-838.png
>
>
> Hi Team,
> I am currently working on data quality POC using griffin.
> So far till now everything is working fine using HIVE as a data source, but
> there is new requirement to add mysql as source.
>
> I have tried every possible way to have *mysql* as custom data connector but
> its not working. Measure is getting created and job is going to successful
> but griffin showing *NO CONTENT* on ui . below is my code for that:-
> "data.sources": [
> {
> "name": "source",
> "connectors": [
> {
> "name": "source1595488803031",
> "type": "CUSTOM",
> "data.unit": "1day",
> "data.time.zone": "",
> "config": {
> "class":
> "org.apache.griffin.measure.datasource.connector.batch.MySqlDataConnector",
> "database": "griffin_poc",
> "tablename": "person_src",
> "url": "jdbc:mysql://griffin:3306/griffin_poc",
> "user": "test_u",
> "password": "test_p",
> "driver": "com.mysql.jdbc.Driver"
> }
> }
> can anyone please provide some valuable suggestions on this , how to use
> mysql as my datasource as it is very critical for my POC. Its an urgent issue.
> Anything i am missing here to link with mysql, please guide me through this.
>
> Also i have setup mysql on ec2 instance using griffin docker image.
>
>
> Any leads will be appreciated.
> Thanks:)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (GRIFFIN-338) Custom Connector for mysql not working
[
https://issues.apache.org/jira/browse/GRIFFIN-338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167430#comment-17167430
]
Chitral Verma edited comment on GRIFFIN-338 at 7/29/20, 6:20 PM:
-
Okay, I understand what's happening.
You're trying to use Griffin along with the frontend and its services. Griffin
was stable a little beyond 0.5.0 release and did not contain implementation for
JDBC that you see in the master branch.The latest version in master branch
however is not yet supported by the services and thus the front end, so I don't
think you will be able to use Master branch for your POC.
Currently, Griffin is in the midst of a complete revamp and I'm doing most of
the heavy lifting regarding this. My suggestion would be to use the code from
an earlier commit, like below, compile it manually and then try using it.
[https://github.com/apache/griffin/tree/48f304dc316ed380fef2c5c7891f7ec41e0ddd30/]
was (Author: chitralverma):
Okay, I understand what's happening.
You're trying to use Griffin along with the frontend and its services. Griffin
was stable a little beyond 0.5.0 release and did not contain implementation for
JDBC that you see in the master branch. Currently, Griffin is in the midst of a
complete revamp and I'm doing most of the heavy lifting regarding this.
My suggestion would be to use the code from an earlier commit, like below,
compile it manually and then try using it.
https://github.com/apache/griffin/tree/48f304dc316ed380fef2c5c7891f7ec41e0ddd30/
> Custom Connector for mysql not working
> --
>
> Key: GRIFFIN-338
> URL: https://issues.apache.org/jira/browse/GRIFFIN-338
> Project: Griffin
> Issue Type: Bug
> Environment: griffin version= 0.5, mysql on EC2 instance using
> griffin docker image
>Reporter: ishan verma
>Priority: Critical
> Attachments: image-2020-07-29-20-23-40-838.png
>
>
> Hi Team,
> I am currently working on data quality POC using griffin.
> So far till now everything is working fine using HIVE as a data source, but
> there is new requirement to add mysql as source.
>
> I have tried every possible way to have *mysql* as custom data connector but
> its not working. Measure is getting created and job is going to successful
> but griffin showing *NO CONTENT* on ui . below is my code for that:-
> "data.sources": [
> {
> "name": "source",
> "connectors": [
> {
> "name": "source1595488803031",
> "type": "CUSTOM",
> "data.unit": "1day",
> "data.time.zone": "",
> "config": {
> "class":
> "org.apache.griffin.measure.datasource.connector.batch.MySqlDataConnector",
> "database": "griffin_poc",
> "tablename": "person_src",
> "url": "jdbc:mysql://griffin:3306/griffin_poc",
> "user": "test_u",
> "password": "test_p",
> "driver": "com.mysql.jdbc.Driver"
> }
> }
> can anyone please provide some valuable suggestions on this , how to use
> mysql as my datasource as it is very critical for my POC. Its an urgent issue.
> Anything i am missing here to link with mysql, please guide me through this.
>
> Also i have setup mysql on ec2 instance using griffin docker image.
>
>
> Any leads will be appreciated.
> Thanks:)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (GRIFFIN-338) Custom Connector for mysql not working
[
https://issues.apache.org/jira/browse/GRIFFIN-338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167430#comment-17167430
]
Chitral Verma commented on GRIFFIN-338:
---
Okay, I understand what's happening.
You're trying to use Griffin along with the frontend and its services. Griffin
was stable a little beyond 0.5.0 release and did not contain implementation for
JDBC that you see in the master branch. Currently, Griffin is in the midst of a
complete revamp and I'm doing most of the heavy lifting regarding this.
My suggestion would be to use the code from an earlier commit, like below,
compile it manually and then try using it.
https://github.com/apache/griffin/tree/48f304dc316ed380fef2c5c7891f7ec41e0ddd30/
> Custom Connector for mysql not working
> --
>
> Key: GRIFFIN-338
> URL: https://issues.apache.org/jira/browse/GRIFFIN-338
> Project: Griffin
> Issue Type: Bug
> Environment: griffin version= 0.5, mysql on EC2 instance using
> griffin docker image
>Reporter: ishan verma
>Priority: Critical
> Attachments: image-2020-07-29-20-23-40-838.png
>
>
> Hi Team,
> I am currently working on data quality POC using griffin.
> So far till now everything is working fine using HIVE as a data source, but
> there is new requirement to add mysql as source.
>
> I have tried every possible way to have *mysql* as custom data connector but
> its not working. Measure is getting created and job is going to successful
> but griffin showing *NO CONTENT* on ui . below is my code for that:-
> "data.sources": [
> {
> "name": "source",
> "connectors": [
> {
> "name": "source1595488803031",
> "type": "CUSTOM",
> "data.unit": "1day",
> "data.time.zone": "",
> "config": {
> "class":
> "org.apache.griffin.measure.datasource.connector.batch.MySqlDataConnector",
> "database": "griffin_poc",
> "tablename": "person_src",
> "url": "jdbc:mysql://griffin:3306/griffin_poc",
> "user": "test_u",
> "password": "test_p",
> "driver": "com.mysql.jdbc.Driver"
> }
> }
> can anyone please provide some valuable suggestions on this , how to use
> mysql as my datasource as it is very critical for my POC. Its an urgent issue.
> Anything i am missing here to link with mysql, please guide me through this.
>
> Also i have setup mysql on ec2 instance using griffin docker image.
>
>
> Any leads will be appreciated.
> Thanks:)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [griffin] chitralverma commented on pull request #575: [GRIFFIN-305] Standardize sink hierarchy
chitralverma commented on pull request #575: URL: https://github.com/apache/griffin/pull/575#issuecomment-665761322 @wankunde @guoyuepeng Can you please review this. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (GRIFFIN-338) Custom Connector for mysql not working
[
https://issues.apache.org/jira/browse/GRIFFIN-338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167287#comment-17167287
]
ishan verma commented on GRIFFIN-338:
-
[~chitralverma] Many thanks for looking into this issue.
I had already used master class JDBC . Pls take a look if this is what you
meant or just using TYPE='jdbc' not custom.
although I have tried all possible scenarios. :(. but its just showing no
content in UI.
!image-2020-07-29-20-23-40-838.png!
> Custom Connector for mysql not working
> --
>
> Key: GRIFFIN-338
> URL: https://issues.apache.org/jira/browse/GRIFFIN-338
> Project: Griffin
> Issue Type: Bug
> Environment: griffin version= 0.5, mysql on EC2 instance using
> griffin docker image
>Reporter: ishan verma
>Priority: Critical
> Attachments: image-2020-07-29-20-22-03-308.png,
> image-2020-07-29-20-23-40-838.png
>
>
> Hi Team,
> I am currently working on data quality POC using griffin.
> So far till now everything is working fine using HIVE as a data source, but
> there is new requirement to add mysql as source.
>
> I have tried every possible way to have *mysql* as custom data connector but
> its not working. Measure is getting created and job is going to successful
> but griffin showing *NO CONTENT* on ui . below is my code for that:-
> "data.sources": [
> {
> "name": "source",
> "connectors": [
> {
> "name": "source1595488803031",
> "type": "CUSTOM",
> "data.unit": "1day",
> "data.time.zone": "",
> "config": {
> "class":
> "org.apache.griffin.measure.datasource.connector.batch.MySqlDataConnector",
> "database": "griffin_poc",
> "tablename": "person_src",
> "url": "jdbc:mysql://griffin:3306/griffin_poc",
> "user": "test_u",
> "password": "test_p",
> "driver": "com.mysql.jdbc.Driver"
> }
> }
> can anyone please provide some valuable suggestions on this , how to use
> mysql as my datasource as it is very critical for my POC. Its an urgent issue.
> Anything i am missing here to link with mysql, please guide me through this.
>
> Also i have setup mysql on ec2 instance using griffin docker image.
>
>
> Any leads will be appreciated.
> Thanks:)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (GRIFFIN-338) Custom Connector for mysql not working
[
https://issues.apache.org/jira/browse/GRIFFIN-338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ishan verma updated GRIFFIN-338:
Attachment: (was: image-2020-07-29-20-22-03-308.png)
> Custom Connector for mysql not working
> --
>
> Key: GRIFFIN-338
> URL: https://issues.apache.org/jira/browse/GRIFFIN-338
> Project: Griffin
> Issue Type: Bug
> Environment: griffin version= 0.5, mysql on EC2 instance using
> griffin docker image
>Reporter: ishan verma
>Priority: Critical
> Attachments: image-2020-07-29-20-23-40-838.png
>
>
> Hi Team,
> I am currently working on data quality POC using griffin.
> So far till now everything is working fine using HIVE as a data source, but
> there is new requirement to add mysql as source.
>
> I have tried every possible way to have *mysql* as custom data connector but
> its not working. Measure is getting created and job is going to successful
> but griffin showing *NO CONTENT* on ui . below is my code for that:-
> "data.sources": [
> {
> "name": "source",
> "connectors": [
> {
> "name": "source1595488803031",
> "type": "CUSTOM",
> "data.unit": "1day",
> "data.time.zone": "",
> "config": {
> "class":
> "org.apache.griffin.measure.datasource.connector.batch.MySqlDataConnector",
> "database": "griffin_poc",
> "tablename": "person_src",
> "url": "jdbc:mysql://griffin:3306/griffin_poc",
> "user": "test_u",
> "password": "test_p",
> "driver": "com.mysql.jdbc.Driver"
> }
> }
> can anyone please provide some valuable suggestions on this , how to use
> mysql as my datasource as it is very critical for my POC. Its an urgent issue.
> Anything i am missing here to link with mysql, please guide me through this.
>
> Also i have setup mysql on ec2 instance using griffin docker image.
>
>
> Any leads will be appreciated.
> Thanks:)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (GRIFFIN-338) Custom Connector for mysql not working
[
https://issues.apache.org/jira/browse/GRIFFIN-338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ishan verma updated GRIFFIN-338:
Attachment: image-2020-07-29-20-23-40-838.png
> Custom Connector for mysql not working
> --
>
> Key: GRIFFIN-338
> URL: https://issues.apache.org/jira/browse/GRIFFIN-338
> Project: Griffin
> Issue Type: Bug
> Environment: griffin version= 0.5, mysql on EC2 instance using
> griffin docker image
>Reporter: ishan verma
>Priority: Critical
> Attachments: image-2020-07-29-20-22-03-308.png,
> image-2020-07-29-20-23-40-838.png
>
>
> Hi Team,
> I am currently working on data quality POC using griffin.
> So far till now everything is working fine using HIVE as a data source, but
> there is new requirement to add mysql as source.
>
> I have tried every possible way to have *mysql* as custom data connector but
> its not working. Measure is getting created and job is going to successful
> but griffin showing *NO CONTENT* on ui . below is my code for that:-
> "data.sources": [
> {
> "name": "source",
> "connectors": [
> {
> "name": "source1595488803031",
> "type": "CUSTOM",
> "data.unit": "1day",
> "data.time.zone": "",
> "config": {
> "class":
> "org.apache.griffin.measure.datasource.connector.batch.MySqlDataConnector",
> "database": "griffin_poc",
> "tablename": "person_src",
> "url": "jdbc:mysql://griffin:3306/griffin_poc",
> "user": "test_u",
> "password": "test_p",
> "driver": "com.mysql.jdbc.Driver"
> }
> }
> can anyone please provide some valuable suggestions on this , how to use
> mysql as my datasource as it is very critical for my POC. Its an urgent issue.
> Anything i am missing here to link with mysql, please guide me through this.
>
> Also i have setup mysql on ec2 instance using griffin docker image.
>
>
> Any leads will be appreciated.
> Thanks:)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (GRIFFIN-338) Custom Connector for mysql not working
[
https://issues.apache.org/jira/browse/GRIFFIN-338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ishan verma updated GRIFFIN-338:
Attachment: image-2020-07-29-20-22-03-308.png
> Custom Connector for mysql not working
> --
>
> Key: GRIFFIN-338
> URL: https://issues.apache.org/jira/browse/GRIFFIN-338
> Project: Griffin
> Issue Type: Bug
> Environment: griffin version= 0.5, mysql on EC2 instance using
> griffin docker image
>Reporter: ishan verma
>Priority: Critical
> Attachments: image-2020-07-29-20-22-03-308.png
>
>
> Hi Team,
> I am currently working on data quality POC using griffin.
> So far till now everything is working fine using HIVE as a data source, but
> there is new requirement to add mysql as source.
>
> I have tried every possible way to have *mysql* as custom data connector but
> its not working. Measure is getting created and job is going to successful
> but griffin showing *NO CONTENT* on ui . below is my code for that:-
> "data.sources": [
> {
> "name": "source",
> "connectors": [
> {
> "name": "source1595488803031",
> "type": "CUSTOM",
> "data.unit": "1day",
> "data.time.zone": "",
> "config": {
> "class":
> "org.apache.griffin.measure.datasource.connector.batch.MySqlDataConnector",
> "database": "griffin_poc",
> "tablename": "person_src",
> "url": "jdbc:mysql://griffin:3306/griffin_poc",
> "user": "test_u",
> "password": "test_p",
> "driver": "com.mysql.jdbc.Driver"
> }
> }
> can anyone please provide some valuable suggestions on this , how to use
> mysql as my datasource as it is very critical for my POC. Its an urgent issue.
> Anything i am missing here to link with mysql, please guide me through this.
>
> Also i have setup mysql on ec2 instance using griffin docker image.
>
>
> Any leads will be appreciated.
> Thanks:)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (GRIFFIN-338) Custom Connector for mysql not working
ishan verma created GRIFFIN-338:
---
Summary: Custom Connector for mysql not working
Key: GRIFFIN-338
URL: https://issues.apache.org/jira/browse/GRIFFIN-338
Project: Griffin
Issue Type: Bug
Environment: griffin version= 0.5, mysql on EC2 instance using
griffin docker image
Reporter: ishan verma
Hi Team,
I am currently working on data quality POC using griffin.
So far till now everything is working fine using HIVE as a data source, but
there is new requirement to add mysql as source.
I have tried every possible way to have *mysql* as custom data connector but
its not working. Measure is getting created and job is going to successful but
griffin showing *NO CONTENT* on ui . below is my code for that:-
"data.sources": [
{
"name": "source",
"connectors": [
{
"name": "source1595488803031",
"type": "CUSTOM",
"data.unit": "1day",
"data.time.zone": "",
"config": {
"class":
"org.apache.griffin.measure.datasource.connector.batch.MySqlDataConnector",
"database": "griffin_poc",
"tablename": "person_src",
"url": "jdbc:mysql://griffin:3306/griffin_poc",
"user": "test_u",
"password": "test_p",
"driver": "com.mysql.jdbc.Driver"
}
}
can anyone please provide some valuable suggestions on this , how to use mysql
as my datasource as it is very critical for my POC. Its an urgent issue.
Anything i am missing here to link with mysql, please guide me through this.
Also i have setup mysql on ec2 instance using griffin docker image.
Any leads will be appreciated.
Thanks:)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [griffin] zgdong1987 commented on pull request #568: [GRIFFIN-323] Refactor configuration Data Source Connector
zgdong1987 commented on pull request #568: URL: https://github.com/apache/griffin/pull/568#issuecomment-665456611 I changed to the version 0.5.0, but failed to compile the UI This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Work logged] (GRIFFIN-323) Refactor configuration for Data Source and Data Source Connector
[
https://issues.apache.org/jira/browse/GRIFFIN-323?focusedWorklogId=464011=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-464011
]
ASF GitHub Bot logged work on GRIFFIN-323:
--
Author: ASF GitHub Bot
Created on: 29/Jul/20 07:53
Start Date: 29/Jul/20 07:53
Worklog Time Spent: 10m
Work Description: zgdong1987 commented on pull request #568:
URL: https://github.com/apache/griffin/pull/568#issuecomment-665456611
I changed to the version 0.5.0, but failed to compile the UI
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 464011)
Time Spent: 1h 50m (was: 1h 40m)
> Refactor configuration for Data Source and Data Source Connector
>
>
> Key: GRIFFIN-323
> URL: https://issues.apache.org/jira/browse/GRIFFIN-323
> Project: Griffin
> Issue Type: Improvement
>Reporter: Chitral Verma
>Assignee: Chitral Verma
>Priority: Major
> Fix For: 0.6.0
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Current config structure for Data Source is as follows,
> {noformat}
>
> "data.sources": [
> {
> "name": "src",
> "connectors": [
> {
> "type": "AVRO",
> "version": "1.7",
> "config": {
> "file.path": "/",
> "file.name": ".avro"
> }
> }
> ]
> },
> {
> "name": "tgt",
> "connectors": [
> {
> "type": "AVRO",
> "version": "1.7",
> "config": {
> "file.path": "/",
> "file.name": ".avro"
> }
> }
> ]
> }
> ]
> {noformat}
>
> This ticket proposes the following changes,
> * remove 'version' from 'DataConnectorParam' as it is not being used
> anywhere in the codebase.
> * change 'connectors' from array type to a single JSON object. Since a data
> source named X may only be of one type (hive, file etc), the connector field
> should not be an array.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [griffin] zgd-2007 removed a comment on pull request #568: [GRIFFIN-323] Refactor configuration Data Source Connector
zgd-2007 removed a comment on pull request #568: URL: https://github.com/apache/griffin/pull/568#issuecomment-665456611 I changed to the version 0.5.0, but failed to compile the UI This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
