[jira] [Comment Edited] (GRIFFIN-145) [Service] Go through service API response standard
[ https://issues.apache.org/jira/browse/GRIFFIN-145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451877#comment-16451877 ] Lionel Liu edited comment on GRIFFIN-145 at 4/27/18 7:23 AM: - refer to [https://developer.mozilla.org/en-US/docs/Web/HTTP/Status] Issues: *For measure* A. Get/update/delete measure by id & get measure by owner/ogr should have similar response. # for a invalid id, will response 404 not found. # for a invalid owner/org, will response 200 ok. B. No validation on critical field while adding measure. # given the input as below, I also get a 201 response which indicates that the measure is acceptable in backend. "measure.type":"griffin", "dq.type":"accuracy4", "process.type":"batch4", C. Update/delete measure response 204 # while updating or deleting a measure successfully, there response 204 with no content. 200 with content should be better D. Get measure names group by org will throw 500 Internal Server Error when some measure has null organization which is not a Essential element of creating a measure *For jobs* A. Geting a invalid job will response 405(Method Not Allowed), why not 404 as well as measure service? B. Adding a invalid job(no measure exist) will response 400(bad request), why not 404? C. Update/delete a job successfully response 204 with no content. 200 with content should be better. *For metrics* Api are not available due to connection exception with 500, haven't tested. *For MetaStore* A. Get table or db metadata have same response for valid/invalid input(table), 200 OK. Why not 404 as well as job/measure for a invalid table? was (Author: evan_wan): refer to [https://developer.mozilla.org/en-US/docs/Web/HTTP/Status] Issues: *For measure* A. Get/update/delete measure by id & get measure by owner/ogr should have similar response. # for a invalid id, will response 404 not found. # for a invalid owner/org, will response 200 ok. B. No validation on critical field while adding measure. # given the input as below, I also get a 201 response which indicates that the measure is acceptable in backend. "measure.type":"griffin", "dq.type":"accuracy4", "process.type":"batch4", C. Update/delete measure response 204 # while updating or deleting a measure successfully, there response 204 with no content. 200 with content should be better D. Get measure names group by org will throw 500 Internal Server Error when some measure has null organization which is not a Essential element of creating a measure *For jobs* A. Geting a invalid job will response 405(Method Not Allowed), why not 404 as well as measure service? B. Adding a invalid job(no measure exist) will response 400(bad request), why not 404? C. Update/delete a job successfully response 204 with no content. 200 with content should be better. *For metrics* Api are not worable due to connection exception with 500, haven't tested. *For MetaStore* A. Get table or db metadata have same response for valid/invalid input(table), 200 OK. Why not 404 as well as job/measure for a invalid table? > [Service] Go through service API response standard > -- > > Key: GRIFFIN-145 > URL: https://issues.apache.org/jira/browse/GRIFFIN-145 > Project: Griffin (Incubating) > Issue Type: Task >Reporter: Lionel Liu >Assignee: wanyin >Priority: Major > Original Estimate: 48h > Remaining Estimate: 48h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (GRIFFIN-145) [Service] Go through service API response standard
[ https://issues.apache.org/jira/browse/GRIFFIN-145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451877#comment-16451877 ] Lionel Liu edited comment on GRIFFIN-145 at 4/27/18 7:24 AM: - refer to [https://developer.mozilla.org/en-US/docs/Web/HTTP/Status] Issues: *For measure* A. Get/update/delete measure by id & get measure by owner/ogr should have similar response. # for a invalid id, will response 404 not found. # for a invalid owner/org, will response 200 ok. B. No validation on critical field while adding measure. # given the input as below, I also get a 201 response which indicates that the measure is acceptable in backend. "measure.type":"griffin", "dq.type":"accuracy4", "process.type":"batch4", C. Update/delete measure response 204 # while updating or deleting a measure successfully, there response 204 with no content. 200 with content should be better D. Get measure names group by org will throw 500 Internal Server Error when some measure has null organization which is not a Essential element of creating a measure *For jobs* A. Getting a invalid job will response 405(Method Not Allowed), why not 404 as well as measure service? B. Adding a invalid job(no measure exist) will response 400(bad request), why not 404? C. Update/delete a job successfully response 204 with no content. 200 with content should be better. *For metrics* Api are not available due to connection exception with 500, haven't tested. *For MetaStore* A. Get table or db metadata have same response for valid/invalid input(table), 200 OK. Why not 404 as well as job/measure for a invalid table? was (Author: evan_wan): refer to [https://developer.mozilla.org/en-US/docs/Web/HTTP/Status] Issues: *For measure* A. Get/update/delete measure by id & get measure by owner/ogr should have similar response. # for a invalid id, will response 404 not found. # for a invalid owner/org, will response 200 ok. B. No validation on critical field while adding measure. # given the input as below, I also get a 201 response which indicates that the measure is acceptable in backend. "measure.type":"griffin", "dq.type":"accuracy4", "process.type":"batch4", C. Update/delete measure response 204 # while updating or deleting a measure successfully, there response 204 with no content. 200 with content should be better D. Get measure names group by org will throw 500 Internal Server Error when some measure has null organization which is not a Essential element of creating a measure *For jobs* A. Geting a invalid job will response 405(Method Not Allowed), why not 404 as well as measure service? B. Adding a invalid job(no measure exist) will response 400(bad request), why not 404? C. Update/delete a job successfully response 204 with no content. 200 with content should be better. *For metrics* Api are not available due to connection exception with 500, haven't tested. *For MetaStore* A. Get table or db metadata have same response for valid/invalid input(table), 200 OK. Why not 404 as well as job/measure for a invalid table? > [Service] Go through service API response standard > -- > > Key: GRIFFIN-145 > URL: https://issues.apache.org/jira/browse/GRIFFIN-145 > Project: Griffin (Incubating) > Issue Type: Task >Reporter: Lionel Liu >Assignee: wanyin >Priority: Major > Original Estimate: 48h > Remaining Estimate: 48h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
suggest to reform the api of measure
Hi guys, now i am playing with griffin in dev env, getting different behavior/response of some api in measure service. Get/update/delete measure by id & get measure by owner/ogr should have similar response. # for a invalid id, will response 404 not found. # for a invalid owner/org, will response 200 ok. in my understanding, we'd better to unify them and reform this. not only in measure, but also in job, scheduler,. etc. thanks, Evan
[jira] [Assigned] (GRIFFIN-145) [Service] Go through service API response standard
[ https://issues.apache.org/jira/browse/GRIFFIN-145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lionel Liu reassigned GRIFFIN-145: -- Assignee: wanyin (was: Lionel Liu) > [Service] Go through service API response standard > -- > > Key: GRIFFIN-145 > URL: https://issues.apache.org/jira/browse/GRIFFIN-145 > Project: Griffin (Incubating) > Issue Type: Task >Reporter: Lionel Liu >Assignee: wanyin >Priority: Major > Original Estimate: 48h > Remaining Estimate: 48h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (GRIFFIN-153) [UI] Can not render accuracy chart on UI
[ https://issues.apache.org/jira/browse/GRIFFIN-153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16456043#comment-16456043 ] ASF GitHub Bot commented on GRIFFIN-153: GitHub user dodobel opened a pull request: https://github.com/apache/incubator-griffin/pull/273 GRIFFIN-153 Fix bugs: Can not render accuracy chart on UI You can merge this pull request into a Git repository by running: $ git pull https://github.com/dodobel/incubator-griffin button Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-griffin/pull/273.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #273 commit 41e0007996ca72684bf7826ee645b08b48615fc0 Author: dodobel <1254288433@...> Date: 2018-01-31T02:54:58Z format code and add scroll bar in measuer view commit 73d0e436277bb2a47591632f97c9decd6823e0cf Author: dodobel <1254288433@...> Date: 2018-01-31T06:49:28Z add scroll bar to sidebar commit 255a3a3151080c0918dec3282f9a5316a935c712 Author: dodobel <1254288433@...> Date: 2018-02-01T05:33:53Z fix bugs commit 6a871fbd51bed535ec89da404d4d48a010808a2f Author: dodobel <1254288433@...> Date: 2018-02-01T05:39:24Z fix conflicts commit 7d9a84d1810fbd7d0d2ad41ba412da8aeb1fe8c2 Author: dodobel <1254288433@...> Date: 2018-02-01T06:21:59Z add scroll to save measure page commit 9f460390e2037391ccc9cd21e513f8615e77f37b Author: dodobel <1254288433@...> Date: 2018-02-05T03:30:05Z show profiling results commit 49d2c5611e19670c47970a2c3ec5694a5b60538a Author: dodobel <1254288433@...> Date: 2018-02-05T03:30:39Z Merge branch 'master' of https://github.com/apache/incubator-griffin commit ea0b36b7807b6da69a07ef8686956780ea77 Author: dodobel <1254288433@...> Date: 2018-02-05T08:13:22Z change style commit da01c8fd03631e369ef498ee1085b14aede5dda3 Author: dodobel <1254288433@...> Date: 2018-02-05T09:59:52Z add button to job page commit b247ffe7e35baa034fbe049967c841e75706a72c Author: dodobel <1254288433@...> Date: 2018-02-05T10:15:30Z fix conflicts commit d4fc8071ee9bc8df818b94f63bb626b57730f60f Author: dodobel <1254288433@...> Date: 2018-02-06T02:35:01Z add time on sidebar commit 6e87edbf64678a96f4a0c24ab689ff8ff56a8850 Author: dodobel <1254288433@...> Date: 2018-02-06T02:37:45Z fix conflicts commit ab2b09231dcb40108373e961e7906577e853c323 Author: dodobel <1254288433@...> Date: 2018-02-06T02:38:50Z Merge branch 'master' of https://github.com/apache/incubator-griffin into button commit f81db358bafb004d56c6aa23692c24cecc27dfb1 Author: dodobel <1254288433@...> Date: 2018-02-07T03:19:25Z Merge branch 'master' of https://github.com/apache/incubator-griffin into button commit e7da55839352dd2a35007dcbe1e3f5cca18cf692 Author: dodobel <1254288433@...> Date: 2018-02-07T07:00:46Z fix bugs commit 046f10641c36432418286fc0b16ca45326efacff Author: dodobel <1254288433@...> Date: 2018-02-08T07:19:05Z show group-by results commit cfca1789642151575382060c9ea7fa0e59f7f164 Author: dodobel <1254288433@...> Date: 2018-02-08T07:21:31Z fix conflicts commit 1a4ad9d249b054912482ec964afe4612662a4834 Author: dodobel <1254288433@...> Date: 2018-02-12T02:38:36Z Merge branch 'master' of https://github.com/apache/incubator-griffin into button commit c46fdd500c7b1a899de35ea0e5662cab54962722 Author: dodobel <1254288433@...> Date: 2018-02-12T02:41:12Z fix bugs commit 71e2e30799fae199eedccce3a95cde3e71207860 Author: dodobel <1254288433@...> Date: 2018-02-23T01:52:06Z fix conflicts commit 8522191b032e0983308ecf2d31eab52a9f8cc217 Author: dodobel <1254288433@...> Date: 2018-02-28T02:44:45Z Merge branch 'master' of https://github.com/apache/incubator-griffin into button commit 152f5169613b90515510cfd305de8838c8ce32fc Author: dodobel <1254288433@...> Date: 2018-02-28T05:07:37Z fix dashboard bugs commit 8cb36222fa218c33284aa1d81bc98edd5275e271 Author: dodobel <1254288433@...> Date: 2018-02-28T06:45:09Z update UI documents commit fe156c20dfc68d3055902cf5c625ef2bf3856176 Author: dodobel <1254288433@...> Date: 2018-03-01T02:28:35Z update document commit f910b533991a056b6e5d998004115c4ac3158efe Author: dodobel <1254288433@...> Date: 2018-03-01T02:28:57Z Merge branch 'master' of https://github.com/apache/incubator-griffin into button commit 1b16fca06c1c5ea82f059849c387a1e8283d13c0 Author: dodobel <1254288433@...> Date: 2018-03-05T03:21:13Z show data desc on measure and job pages commit e188e516e4ae2eddcbc429d98e4d08d4f71a0211 Author: dodobel <1254288433@...> Date: 2018-03-05T03:21:26Z Merge branch 'master' of https://github.com/apache/incubator-griffin into button commit 26b2d5d0c19ebe585133464ad38aa199f2841a8e Author: dodobel
[jira] [Resolved] (GRIFFIN-28) Contribute to Griffin
[ https://issues.apache.org/jira/browse/GRIFFIN-28?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Guo resolved GRIFFIN-28. Resolution: Fixed > Contribute to Griffin > - > > Key: GRIFFIN-28 > URL: https://issues.apache.org/jira/browse/GRIFFIN-28 > Project: Griffin (Incubating) > Issue Type: Wish >Reporter: Abhishek Mynam >Assignee: William Guo >Priority: Minor > Labels: newbie > > Hi Griffin team, > My name is Abhi. I am a software engineer working for KPMG LLP. I am always > looking to build something, whether it's an application, an API or a library. > I started couple of projects but I did not complete them. As I started > working deep in those projects I realized that the projects I am picking are > not very innovative and they are not special. I felt that the best way to > build something and learn on how to think and innovate is to start > contributing for open source projects. In that research I came across your > Apache foundation project - Griffin. The area of your research is one of my > areas of interest and I am very impressed by the project. I would like to > check if I can get an opportunity to work for your project. > Below are my LinkedIn profile and GitHub repository. > https://www.linkedin.com/in/abhishekmynam > https://github.com/abhishekmynam > Thanks & Regards, > Abhishek Mynam > email : abhishek.my...@gmail.com -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (GRIFFIN-145) [Service] Go through service API response standard
[ https://issues.apache.org/jira/browse/GRIFFIN-145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lionel Liu updated GRIFFIN-145: --- Sprint: Apache Sprint 2, Apache Sprint 3 (was: Apache Sprint 2) > [Service] Go through service API response standard > -- > > Key: GRIFFIN-145 > URL: https://issues.apache.org/jira/browse/GRIFFIN-145 > Project: Griffin (Incubating) > Issue Type: Task >Reporter: Lionel Liu >Assignee: wanyin >Priority: Major > Original Estimate: 48h > Remaining Estimate: 48h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (GRIFFIN-117) [Service] Provide api for users to download missing records in accuracy
[ https://issues.apache.org/jira/browse/GRIFFIN-117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lionel Liu updated GRIFFIN-117: --- Sprint: Apache Sprint 2, Apache Sprint 3 (was: Apache Sprint 2) > [Service] Provide api for users to download missing records in accuracy > --- > > Key: GRIFFIN-117 > URL: https://issues.apache.org/jira/browse/GRIFFIN-117 > Project: Griffin (Incubating) > Issue Type: New Feature >Reporter: Lionel Liu >Assignee: Zhao, Vincent >Priority: Major > Original Estimate: 48h > Remaining Estimate: 48h > > In accuracy, users want to download the missing records, we need to provide > such api for them. > # user can get the missing records list of each metric point. > # user can get the url of the specific missing records. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (GRIFFIN-123) [Measure] Code review of measure module, refactor and enhance the code style
[ https://issues.apache.org/jira/browse/GRIFFIN-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lionel Liu updated GRIFFIN-123: --- Sprint: Apache Sprint 2, Apache Sprint 3 (was: Apache Sprint 2) > [Measure] Code review of measure module, refactor and enhance the code style > > > Key: GRIFFIN-123 > URL: https://issues.apache.org/jira/browse/GRIFFIN-123 > Project: Griffin (Incubating) > Issue Type: Improvement >Reporter: Lionel Liu >Assignee: William Guo >Priority: Major > Original Estimate: 72h > Remaining Estimate: 72h > > Code review of measure module, refactor and enhance the code style -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (GRIFFIN-103) [Griffin] Release of incubating-0.2.0
[ https://issues.apache.org/jira/browse/GRIFFIN-103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lionel Liu updated GRIFFIN-103: --- Sprint: Apache Sprint 1, Apache Sprint 2, Apache Sprint 3 (was: Apache Sprint 1, Apache Sprint 2) > [Griffin] Release of incubating-0.2.0 > - > > Key: GRIFFIN-103 > URL: https://issues.apache.org/jira/browse/GRIFFIN-103 > Project: Griffin (Incubating) > Issue Type: Task >Reporter: Lionel Liu >Assignee: Lionel Liu >Priority: Major > Fix For: 0.2.0-incubating > > Original Estimate: 72h > Remaining Estimate: 72h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (GRIFFIN-114) [Service][UT] Enhance unit test of config, info, util in service module
[ https://issues.apache.org/jira/browse/GRIFFIN-114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lionel Liu updated GRIFFIN-114: --- Sprint: Apache Sprint 2, Apache Sprint 3 (was: Apache Sprint 2) > [Service][UT] Enhance unit test of config, info, util in service module > --- > > Key: GRIFFIN-114 > URL: https://issues.apache.org/jira/browse/GRIFFIN-114 > Project: Griffin (Incubating) > Issue Type: Task >Reporter: Lionel Liu >Assignee: wanyin >Priority: Major > Original Estimate: 24h > Remaining Estimate: 24h > > In service module, UT of config, info and util need to be enhanced. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (GRIFFIN-153) [UI] Can not render accuracy chart on UI
[ https://issues.apache.org/jira/browse/GRIFFIN-153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lionel Liu updated GRIFFIN-153: --- Sprint: Apache Sprint 2, Apache Sprint 3 (was: Apache Sprint 2) > [UI] Can not render accuracy chart on UI > > > Key: GRIFFIN-153 > URL: https://issues.apache.org/jira/browse/GRIFFIN-153 > Project: Griffin (Incubating) > Issue Type: Bug >Reporter: Lionel Liu >Assignee: Yuqin Xuan >Priority: Major > > On UI, no matter the dashboard page or side bar, it can not render the > accuracy chart in this situation: > I created an accuracy measure, and created two jobs for this measure, one is > daily, and another is hourly. > After some hours, the hourly job has triggered several times, but the daily > job hasn't started yet. > I created the daily job earlier than the hourly job, in this situation, in > the dashboard page or side bar, I can not see any chart of my accuracy > measure. But it should be able to render the hourly job chart, because the > metric data has been calculated. > > The problem seems like: > When ui get the metrics data like this: > > {code:java} > { > "accu":[ > { > "name":"accu_daily", > "type":"accuracy", > "metricValues":[] > }, > { > "name":"accu_hourly", > "type":"accuracy", > "metricValues":[ > { > "name":"accu_hourly", > "tmst":152470080, > "value":{ > "total":125000, > "miss":485, > "matched":124515 > } > } > ] > } > ] > }{code} > It should be able to render a chart named "accu_hourly" under the measure > "accu", but it could not render anything. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (GRIFFIN-148) [Doc] Use case document of crawler
[ https://issues.apache.org/jira/browse/GRIFFIN-148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lionel Liu updated GRIFFIN-148: --- Sprint: Apache Sprint 2, Apache Sprint 3 (was: Apache Sprint 2) > [Doc] Use case document of crawler > -- > > Key: GRIFFIN-148 > URL: https://issues.apache.org/jira/browse/GRIFFIN-148 > Project: Griffin (Incubating) > Issue Type: Task >Reporter: Lionel Liu >Assignee: William Guo >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (GRIFFIN-28) Contribute to Griffin
[ https://issues.apache.org/jira/browse/GRIFFIN-28?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16456059#comment-16456059 ] William Guo commented on GRIFFIN-28: https://cwiki.apache.org/confluence/display/GRIFFIN/0.+Development > Contribute to Griffin > - > > Key: GRIFFIN-28 > URL: https://issues.apache.org/jira/browse/GRIFFIN-28 > Project: Griffin (Incubating) > Issue Type: Wish >Reporter: Abhishek Mynam >Assignee: William Guo >Priority: Minor > Labels: newbie > > Hi Griffin team, > My name is Abhi. I am a software engineer working for KPMG LLP. I am always > looking to build something, whether it's an application, an API or a library. > I started couple of projects but I did not complete them. As I started > working deep in those projects I realized that the projects I am picking are > not very innovative and they are not special. I felt that the best way to > build something and learn on how to think and innovate is to start > contributing for open source projects. In that research I came across your > Apache foundation project - Griffin. The area of your research is one of my > areas of interest and I am very impressed by the project. I would like to > check if I can get an opportunity to work for your project. > Below are my LinkedIn profile and GitHub repository. > https://www.linkedin.com/in/abhishekmynam > https://github.com/abhishekmynam > Thanks & Regards, > Abhishek Mynam > email : abhishek.my...@gmail.com -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re:Livy Error
Hi Karan, I need some more information, how did you configure in sparkJob.properties? Can you post to livy directly by curl? Maybe this will help you: https://stackoverflow.com/questions/46909048/livy-rest-api-get-requests-work-but-post-requests-fail-with-401-authentication -- Regards, Lionel, Liu At 2018-04-27 17:37:02, "Karan Gupta"wrote: Hi Lionel, I am trying to submit a spark job but the job does not gets submitted and I see the following error message on the console at the scheduled time. 2018-04-27 05:35:00.278 ERROR 106533 --- [ryBean_Worker-5] o.a.griffin.core.job.SparkSubmitJob : Post to livy error. 401 Authentication required Could you help me out with the same. Thank you, Karan Gupta Any comments or statements made in this email are not necessarily those of Tavant Technologies. The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. If you have received this in error, please contact the sender and delete the material from any computer. All emails sent from or to Tavant Technologies may be subject to our monitoring procedures.
[GitHub] incubator-griffin pull request #272: GRIFFIN-153
Github user dodobel closed the pull request at: https://github.com/apache/incubator-griffin/pull/272 ---