[GitHub] [incubator-hivemall] Solodye opened a new pull request #192: UDF map_roulette() created

[GitBox]

Solodye opened a new pull request #192: UDF map_roulette() created
URL: https://github.com/apache/incubator-hivemall/pull/192
 
 
   ## What changes were proposed in this pull request?
   
   Create 
   
   ## What type of PR is it?
   
   Feature and documentation
   
   ## What is the Jira issue?
   
   
https://issues.apache.org/jira/projects/HIVEMALL/issues/HIVEMALL-253?filter=allopenissues
   [HIVEMALL-253]
   
   ## How was this patch tested?
   
   Unit test and handmade test:
   ```sql
   -- 1. Empty map
   select map_roulette(map()); -- NULL
   -- 2. The map that key is null
   select map_roulette(map(null, null, null, null)); -- NULL
   -- 3. A map only one entry
   select map_roulette(map(1, 2)); -- 1
   -- 4. A map with normal value (1, 1) and null value (null, null)
   select map_roulette(map(1, 2, null, null)); -- 50% 1, 50% null
   -- 5. A map contains zero weight
   select map_roulette(map(1, 2, 3, 0)); -- 1, but 3 may appear at a very low 
probability
   -- 6. All weights are zero
   select map_roulette(map(1, 0)); -- NULL
   select map_roulette(map(1, 0, '5', 0)); -- NULL
   -- 7. The testcase close to real data
   select map_roulette(map(1, 0.5, 'Wang', 0.5)); -- 50% Wang, 50% 1
   select map_roulette(map(1, 0.5, 'Wang', null)); -- 50% Wang, 50% 1
   select map_roulette(map(1, 0.5, 'Wang', null, 'Zhang', null)); -- 1/3 Wang, 
1/3 1, 1/3 Zhang
   select map_roulette(map(1, null, 'Wang', null)); -- NULL
   select map_roulette(map(1, 0, 'Wang', 0, 'Zhang', 1)); -- Zhang， but 'Wang', 
'Zhang' may appear at a very low probability
   select map_roulette(map('Wang', 3, 'Zhang', 2, 'Li', 6, 'Zhao', 8, 'Tom', 
10)); -- 3/29 Wang, 2/29 Zhang, 6/29 Li, 8/29 Zhao, 10/29 Tom
   -- 8. A usecase with wrong input
   select map_roulette(map('Wong', 'A string', 'Zhao', 2));
   -- Failed with exception 
java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: Error 
evaluating map_roulette([map('Wong':'A string','Zhao':2)])
   -- Because weight must be a number.
   -- 9. Negative weight
   select map_roulette(map('Wong', 3, 'Zhao', -2));
   -- Failed with exception 
java.io.IOException:org.apache.hadoop.hive.ql.exec.UDFArgumentException: -2 < 0
   -- 10. Use this function with to_map()
   select map_roulette(to_map(a, b)) -- 25% Tom, 21% Zhang, 54% Wang
   from(
   select 'Wang' as a, 54 as b
   union
   select 'Zhang' as a, 21 as b
   union
   select 'Tom' as a, 25 as b
   )tmp;
   ```
   
   ## How to use this feature?
   
   In doc: docs/gitbook/misc/generic_funcs.md
   No. 10  handmade test is the best practice. 
   
   ## Checklist
   
   - [x] Did you apply source code formatter, i.e., `./bin/format_code.sh`, for 
your commit?
   - [x] Did you run system tests on Hive (or Spark)?
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services