GitHub user bhlx3lyx7 opened a pull request:

    https://github.com/apache/incubator-griffin/pull/254

    [GRIFFIN-135] support completeness measurement for batch and streaming mode

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/bhlx3lyx7/incubator-griffin tmst

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-griffin/pull/254.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #254
    
----
commit 2f1243c36629f98ba4e2f0480da25414c5717cfb
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-05T04:36:03Z

    test

commit 982d1742b96d4ce7d4ba5e1324db92a9296f8e79
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-05T08:02:40Z

    pass batch test

commit 2f36ab653d0efc53fe223389a1a21cb1dc702b29
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-07T06:34:51Z

    tmstNameOpt

commit c6a5650f1a2410ac26c97cddc96afaecdd54a67d
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-07T08:52:41Z

    hdfs persist and evaluate.rule

commit 26a235e113b947431f1c262cbf3aed31f8816549
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-07T08:54:09Z

    merge conflict

commit a7e28bb5cb6f2ecae4cec706b9c5a5d7cf29a4aa
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-07T13:25:13Z

    pass batch, waiting for test speed

commit 6da53e965b931b722953abf385964b91c74f2afc
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-08T02:43:18Z

    pre-proc done

commit 642d4a3575a5b1e3843bf5337fd16e95ee3dfaa9
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-11T09:43:57Z

    hdfs persist enhance

commit 472a1f998936b476becc685a7a57b1ce680108a0
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-13T02:53:32Z

    register table

commit 3a00dee836693f3a4345c101f941b590eb516277
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-13T10:02:13Z

    manage temp tables, waiting for ignore cache group

commit ce4558d17f15cf370ab36f707801919fb2939ddc
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-14T02:04:53Z

    enable ignore cache for accuracy opr

commit ff5797f627058f81da5135cc0a70c01e0870b98b
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-14T02:19:32Z

    clear metrics internal columns

commit f44396f5bc5f04ad80e828d450a1798e083e2149
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-14T02:21:30Z

    Merge branch 'tmst' into tmst-crawler

commit 651cf3e8ffebe8e2981e0c993f4f9d1f00457878
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-14T03:16:28Z

    hdfs

commit f3e81b1261b00fc09c9b1703662c26b9ec969c08
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-15T06:19:42Z

    not done

commit 08bd2242b7db577c902932a0894d133ffc9551ff
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-18T05:25:03Z

    optimize accuracy

commit ca0e8c264cdc457fe98532545d99ebaa11e86f14
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-18T10:11:53Z

    opt accuracy

commit 0b664747efaaabb3e2ebd68f6adf1b31fad0f671
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-20T08:37:40Z

    performance bad

commit 725ee4009da27bf694c4f932fac5def690aedc8b
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-24T14:29:50Z

    define config json, need to treat metric and record persist as extra 
process outside of pipeline, batch and streaming with different rule adaptor, 
to treat tmst column

commit 5a5dff6a54f77aaaadc7db6d1414643c3255580c
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-24T14:42:37Z

    streaming accuracy config json update, persist processes in batch and 
streaming mode are different, considering take different process in data source 
pre-process

commit f2ad36e4a0821a1e454fa3ef8a7da27c349c129e
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-25T02:01:11Z

    update profiling streaming spark sql config json

commit 8d55964f0e9c41b56a378c98ab3d7592294b357f
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-26T14:18:19Z

    pass batch

commit b28579b49686a52680b0974d8086710457a923c8
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-26T15:11:26Z

    wait for persist

commit bd7c886c63dba36ca8435fc5518242d00aea8994
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-26T15:25:52Z

    multi thread persist

commit 0d9f8198ad71509c8b54ec82d50f4c5852cb7acb
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-27T10:34:00Z

    streaming pass

commit b86612a86456bbff6d005865486237333aa5989a
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-28T01:44:25Z

    hdfs util

commit 0e1f9681b5293c270ed9245b3416fadbc9ad88ab
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-28T04:58:47Z

    add df.cache in dq engine, to fix df reuse bug if file removed before the 
lasy execution

commit 9357b52dd12de3cd4e54655b32e1d6643131a4ca
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-28T07:10:18Z

    persist modify to iterable for streaming mode

commit 0cd121b386e15da7373bb2302a4a0783c1b49b50
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-28T12:25:32Z

    fix all matched ignore bug

commit 84892a8e7787bbd644b729e5ca221d2e4b521780
Author: Lionel Liu <bhlx3lyx7@...>
Date:   2017-12-29T05:31:19Z

    enable accuracy df opr

----


---

Reply via email to