[
https://issues.apache.org/jira/browse/GRIFFIN-135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431707#comment-16431707
]
ASF GitHub Bot commented on GRIFFIN-135:
----------------------------------------
GitHub user bhlx3lyx7 opened a pull request:
https://github.com/apache/incubator-griffin/pull/254
[GRIFFIN-135] support completeness measurement for batch and streaming mode
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bhlx3lyx7/incubator-griffin tmst
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-griffin/pull/254.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #254
----
commit 2f1243c36629f98ba4e2f0480da25414c5717cfb
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-05T04:36:03Z
test
commit 982d1742b96d4ce7d4ba5e1324db92a9296f8e79
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-05T08:02:40Z
pass batch test
commit 2f36ab653d0efc53fe223389a1a21cb1dc702b29
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-07T06:34:51Z
tmstNameOpt
commit c6a5650f1a2410ac26c97cddc96afaecdd54a67d
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-07T08:52:41Z
hdfs persist and evaluate.rule
commit 26a235e113b947431f1c262cbf3aed31f8816549
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-07T08:54:09Z
merge conflict
commit a7e28bb5cb6f2ecae4cec706b9c5a5d7cf29a4aa
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-07T13:25:13Z
pass batch, waiting for test speed
commit 6da53e965b931b722953abf385964b91c74f2afc
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-08T02:43:18Z
pre-proc done
commit 642d4a3575a5b1e3843bf5337fd16e95ee3dfaa9
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-11T09:43:57Z
hdfs persist enhance
commit 472a1f998936b476becc685a7a57b1ce680108a0
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-13T02:53:32Z
register table
commit 3a00dee836693f3a4345c101f941b590eb516277
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-13T10:02:13Z
manage temp tables, waiting for ignore cache group
commit ce4558d17f15cf370ab36f707801919fb2939ddc
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-14T02:04:53Z
enable ignore cache for accuracy opr
commit ff5797f627058f81da5135cc0a70c01e0870b98b
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-14T02:19:32Z
clear metrics internal columns
commit f44396f5bc5f04ad80e828d450a1798e083e2149
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-14T02:21:30Z
Merge branch 'tmst' into tmst-crawler
commit 651cf3e8ffebe8e2981e0c993f4f9d1f00457878
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-14T03:16:28Z
hdfs
commit f3e81b1261b00fc09c9b1703662c26b9ec969c08
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-15T06:19:42Z
not done
commit 08bd2242b7db577c902932a0894d133ffc9551ff
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-18T05:25:03Z
optimize accuracy
commit ca0e8c264cdc457fe98532545d99ebaa11e86f14
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-18T10:11:53Z
opt accuracy
commit 0b664747efaaabb3e2ebd68f6adf1b31fad0f671
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-20T08:37:40Z
performance bad
commit 725ee4009da27bf694c4f932fac5def690aedc8b
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-24T14:29:50Z
define config json, need to treat metric and record persist as extra
process outside of pipeline, batch and streaming with different rule adaptor,
to treat tmst column
commit 5a5dff6a54f77aaaadc7db6d1414643c3255580c
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-24T14:42:37Z
streaming accuracy config json update, persist processes in batch and
streaming mode are different, considering take different process in data source
pre-process
commit f2ad36e4a0821a1e454fa3ef8a7da27c349c129e
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-25T02:01:11Z
update profiling streaming spark sql config json
commit 8d55964f0e9c41b56a378c98ab3d7592294b357f
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-26T14:18:19Z
pass batch
commit b28579b49686a52680b0974d8086710457a923c8
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-26T15:11:26Z
wait for persist
commit bd7c886c63dba36ca8435fc5518242d00aea8994
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-26T15:25:52Z
multi thread persist
commit 0d9f8198ad71509c8b54ec82d50f4c5852cb7acb
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-27T10:34:00Z
streaming pass
commit b86612a86456bbff6d005865486237333aa5989a
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-28T01:44:25Z
hdfs util
commit 0e1f9681b5293c270ed9245b3416fadbc9ad88ab
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-28T04:58:47Z
add df.cache in dq engine, to fix df reuse bug if file removed before the
lasy execution
commit 9357b52dd12de3cd4e54655b32e1d6643131a4ca
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-28T07:10:18Z
persist modify to iterable for streaming mode
commit 0cd121b386e15da7373bb2302a4a0783c1b49b50
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-28T12:25:32Z
fix all matched ignore bug
commit 84892a8e7787bbd644b729e5ca221d2e4b521780
Author: Lionel Liu <bhlx3lyx7@...>
Date: 2017-12-29T05:31:19Z
enable accuracy df opr
----
> [Measure] Support Completeness measurement as a new feature
> -----------------------------------------------------------
>
> Key: GRIFFIN-135
> URL: https://issues.apache.org/jira/browse/GRIFFIN-135
> Project: Griffin (Incubating)
> Issue Type: New Feature
> Reporter: Lionel Liu
> Assignee: Lionel Liu
> Priority: Major
> Fix For: 1.0.0-incubating
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> Completeness measures the absence of blank (null or empty string) values or
> the presence of non-blank values.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
