GitHub user takuti opened a pull request:

    [HIVEMALL-191] Add Kryo serialization test to existing workaround code

    ## What changes were proposed in this pull request?
    Add Kryo serialization test to existing workaround code as:
    To be more precise, currently two UDFs `quantified_features` and 
`tokenize_ja` explicitly have the workaround lazy instantiation code. So, this 
PR makes their `transient` keyword unnecessary.
    ## What type of PR is it?
    ## What is the Jira issue?
    ## How was this patch tested?
    Added some unit tests, and manually tested as well
    ## Checklist
    - [x] Did you apply source code formatter, i.e., `mvn formatter:format`, 
for your commit?
    - [x] Did you run system tests on Hive (or Spark)?

You can merge this pull request into a Git repository by running:

    $ git pull HIVEMALL-191

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #145
commit 554cd46a84875836733fd55aae0c67746f7aa9dc
Author: Takuya Kitazawa <k.takuti@...>
Date:   2018-04-16T03:57:02Z

    Refactor and add test to QuantifiedFeaturesUDTF
    Test case checks Kryo serialization as well.

commit bb6d5cb4d98dd1cc57e2bb6afe8f74b2ff4002ee
Author: Takuya Kitazawa <k.takuti@...>
Date:   2018-04-16T05:12:40Z

    Add TestUtils to NLP module

commit fc676513c5cec08013d723d4ea5e2003506cb8b6
Author: Takuya Kitazawa <k.takuti@...>
Date:   2018-04-16T05:13:08Z

    Update Kuromoji analyzer initialization logic for Kryo serialization
    `initialize()` creates Java standard objects, and UDF stores them as
    instance variables. Kuromoji-specific externally-defined objects (e.g.,
    CharArraySet) will be created in `evaluate()` at the time of
    initializing Kuromoji analyzer.
    This modification prevents unexpectable KryoException caused by
    instantiation of Kuromoji-specific objects.

commit f2ca8ee83c58d0b86a8f479b3432528e4c79486a
Author: Takuya Kitazawa <k.takuti@...>
Date:   2018-04-16T05:22:00Z

    Make argument of SmartcnUDF.stopWords nullable

commit 6f9193cfa8c29523a127a12441fe00a9b1d87906
Author: Takuya Kitazawa <k.takuti@...>
Date:   2018-04-16T05:22:24Z

    Add serialization test to SmartcnUDFTest



Reply via email to