GitHub user takuti opened a pull request:
https://github.com/apache/incubator-hivemall/pull/145
[HIVEMALL-191] Add Kryo serialization test to existing workaround code
## What changes were proposed in this pull request?
Add Kryo serialization test to existing workaround code as:
https://github.com/apache/incubator-hivemall/commit/f6765dff7be67e1a3327709bbb9bfdc6eba7b97f
To be more precise, currently two UDFs `quantified_features` and
`tokenize_ja` explicitly have the workaround lazy instantiation code. So, this
PR makes their `transient` keyword unnecessary.
## What type of PR is it?
Improvement
## What is the Jira issue?
https://issues.apache.org/jira/browse/HIVEMALL-191
## How was this patch tested?
Added some unit tests, and manually tested as well
## Checklist
- [x] Did you apply source code formatter, i.e., `mvn formatter:format`,
for your commit?
- [x] Did you run system tests on Hive (or Spark)?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/takuti/incubator-hivemall HIVEMALL-191
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-hivemall/pull/145.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #145
----
commit 554cd46a84875836733fd55aae0c67746f7aa9dc
Author: Takuya Kitazawa <k.takuti@...>
Date: 2018-04-16T03:57:02Z
Refactor and add test to QuantifiedFeaturesUDTF
Test case checks Kryo serialization as well.
commit bb6d5cb4d98dd1cc57e2bb6afe8f74b2ff4002ee
Author: Takuya Kitazawa <k.takuti@...>
Date: 2018-04-16T05:12:40Z
Add TestUtils to NLP module
commit fc676513c5cec08013d723d4ea5e2003506cb8b6
Author: Takuya Kitazawa <k.takuti@...>
Date: 2018-04-16T05:13:08Z
Update Kuromoji analyzer initialization logic for Kryo serialization
`initialize()` creates Java standard objects, and UDF stores them as
instance variables. Kuromoji-specific externally-defined objects (e.g.,
CharArraySet) will be created in `evaluate()` at the time of
initializing Kuromoji analyzer.
This modification prevents unexpectable KryoException caused by
instantiation of Kuromoji-specific objects.
commit f2ca8ee83c58d0b86a8f479b3432528e4c79486a
Author: Takuya Kitazawa <k.takuti@...>
Date: 2018-04-16T05:22:00Z
Make argument of SmartcnUDF.stopWords nullable
commit 6f9193cfa8c29523a127a12441fe00a9b1d87906
Author: Takuya Kitazawa <k.takuti@...>
Date: 2018-04-16T05:22:24Z
Add serialization test to SmartcnUDFTest
----
---
