[
https://issues.apache.org/jira/browse/PIG-4947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Youngjin updated PIG-4947:
--------------------------
Description:
LOAD with HBaseStorage using a mix of pure wildcards and prefixed wildcards
results in empty maps for the pure wildcarded column families.
Here is my test scenario.
~~ Create test HBASE table.~~
create 'test_sha1', '3', 'i', 'd'
put 'test_sha1', '1', '3:name', 'youngjin'
put 'test_sha1', '1', 'i:whatever', 'true'
put 'test_sha1', '1', 'd:forgemeta_1_whatever', 'true'
---
1. If use pure wildcarded column families then it works.
sha1_contents_succeed = load 'hbase://test_sha1' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('3:* i:* d:*', '-loadKey
true') as (row_key:chararray, unpacker:map[], stats_i:map[], forgemeta:map[]);
dump sha1_contents_succeed;
(1,[name#youngjin],[whatever#true],[forgemeta_1_whatever#true])
2. If use a mix of pure wildcards and prefixed wildcards - When only one column
family has prefixed wildcards - then result but only shows a prefixed wildcards
column family.
sha1_contents_no_result = load 'hbase://test_sha1' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('3:* i:* d:forgemeta_1_*',
'-loadKey true') as (row_key:chararray, unpacker:map[], stats_i:map[],
forgemeta:map[]);
dump sha1_contents_no_result;
===> (1,[],[],[forgemeta_1_whatever#true])
If use a mix of pure wildcards and prefixed wildcards - When only one column
family has prefixed wildcards - then result but only shows a prefixed wildcards
column family.
sha1_contents_no_result = load 'hbase://test_sha1' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('3:na* i:* d:*', '-loadKey
true') as (row_key:chararray, unpacker:map[], stats_i:map[], forgemeta:map[]);
(1,[name#youngjin],[],[])
If use a mix of pure wildcards and prefixed wildcards - When only one column
family has prefixed wildcards - then result but only shows a prefixed wildcards
column family.
sha1_contents_no_result = load 'hbase://test_sha1' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('3:* i:wh* d:*', '-loadKey
true') as (row_key:chararray, unpacker:map[], stats_i:map[], forgemeta:map[]);
(1,[],[whatever#true],[])
was:
LOAD with HBaseStorage using a mix of pure wildcards and prefixed wildcards
results in empty maps for the pure wildcarded column families.
Here is my test scenario.
~~ Create test HBASE table.~~
create 'test_sha1', '3', 'i', 'd'
put 'test_sha1', '1', '3:name', 'youngjin'
put 'test_sha1', '1', 'i:whatever', 'true'
put 'test_sha1', '1', 'd:forgemeta_1_whatever', 'true'
---
1. If use pure wildcarded column families then it works.
sha1_contents_succeed = load 'hbase://test_sha1' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('3:* i:* d:*', '-loadKey
true') as (row_key:chararray, unpacker:map[], stats_i:map[], forgemeta:map[]);
dump sha1_contents_succeed;
(1,[name#youngjin],[whatever#true],[forgemeta_1_whatever#true])
2. If use a mix of pure wildcards and prefixed wildcards - When the last column
families has prefixed wildcards - then result but only shows a prefixed
wildcards column family.
sha1_contents_no_result = load 'hbase://test_sha1' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('3:* i:* d:forgemeta_1_*',
'-loadKey true') as (row_key:chararray, unpacker:map[], stats_i:map[],
forgemeta:map[]);
dump sha1_contents_no_result;
===> (1,[],[],[forgemeta_1_whatever#true])
If use a mix of pure wildcards and prefixed wildcards - When use a prefixed
wildcards at the first or second column families.. then result but only shows a
prefixed wildcards column family.
sha1_contents_no_result = load 'hbase://test_sha1' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('3:na* i:* d:*', '-loadKey
true') as (row_key:chararray, unpacker:map[], stats_i:map[], forgemeta:map[]);
(1,[name#youngjin],[],[])
sha1_contents_no_result = load 'hbase://test_sha1' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('3:* i:wh* d:*', '-loadKey
true') as (row_key:chararray, unpacker:map[], stats_i:map[], forgemeta:map[]);
(1,[],[whatever#true],[])
> LOAD with HBaseStorage using a mix of pure wildcards and prefixed wildcards
> results in empty maps for the pure wildcarded column families.
> ------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-4947
> URL: https://issues.apache.org/jira/browse/PIG-4947
> Project: Pig
> Issue Type: Bug
> Components: grunt
> Affects Versions: 0.15.0
> Environment: HDP 2.4.0
> Pig 0.15.0
> Reporter: Youngjin
> Assignee: Daniel Dai
> Attachments: PIG-4947-1.patch
>
>
> LOAD with HBaseStorage using a mix of pure wildcards and prefixed wildcards
> results in empty maps for the pure wildcarded column families.
> Here is my test scenario.
> ~~ Create test HBASE table.~~
> create 'test_sha1', '3', 'i', 'd'
> put 'test_sha1', '1', '3:name', 'youngjin'
> put 'test_sha1', '1', 'i:whatever', 'true'
> put 'test_sha1', '1', 'd:forgemeta_1_whatever', 'true'
> ---
> 1. If use pure wildcarded column families then it works.
> sha1_contents_succeed = load 'hbase://test_sha1' using
> org.apache.pig.backend.hadoop.hbase.HBaseStorage('3:* i:* d:*', '-loadKey
> true') as (row_key:chararray, unpacker:map[], stats_i:map[],
> forgemeta:map[]);
> dump sha1_contents_succeed;
> (1,[name#youngjin],[whatever#true],[forgemeta_1_whatever#true])
> 2. If use a mix of pure wildcards and prefixed wildcards - When only one
> column family has prefixed wildcards - then result but only shows a prefixed
> wildcards column family.
> sha1_contents_no_result = load 'hbase://test_sha1' using
> org.apache.pig.backend.hadoop.hbase.HBaseStorage('3:* i:* d:forgemeta_1_*',
> '-loadKey true') as (row_key:chararray, unpacker:map[], stats_i:map[],
> forgemeta:map[]);
> dump sha1_contents_no_result;
> ===> (1,[],[],[forgemeta_1_whatever#true])
> If use a mix of pure wildcards and prefixed wildcards - When only one column
> family has prefixed wildcards - then result but only shows a prefixed
> wildcards column family.
> sha1_contents_no_result = load 'hbase://test_sha1' using
> org.apache.pig.backend.hadoop.hbase.HBaseStorage('3:na* i:* d:*', '-loadKey
> true') as (row_key:chararray, unpacker:map[], stats_i:map[],
> forgemeta:map[]);
> (1,[name#youngjin],[],[])
> If use a mix of pure wildcards and prefixed wildcards - When only one column
> family has prefixed wildcards - then result but only shows a prefixed
> wildcards column family.
> sha1_contents_no_result = load 'hbase://test_sha1' using
> org.apache.pig.backend.hadoop.hbase.HBaseStorage('3:* i:wh* d:*', '-loadKey
> true') as (row_key:chararray, unpacker:map[], stats_i:map[],
> forgemeta:map[]);
> (1,[],[whatever#true],[])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
