[
https://issues.apache.org/jira/browse/PIG-833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745140#action_12745140
]
Jing Huang commented on PIG-833:
--------------------------------
Here is an example for different data types:
final static String STR_SCHEMA = "s1:bool, s2:int, s3:long, s4:float,
s5:string, s6:bytes, r1:record(f1:int, f2:long), r2:record(r3:record(f3:float,
f4)), m1:map(string),m2:map(map(int)), c:collection(f13:double, f14:float,
f15:bytes)";
final static String STR_STORAGE = "[s1, s2]; [m1#{a}]; [r1.f1]; [s3, s4,
r2.r3.f3]; [s5, s6, m2#{x|y}]; [r1.f2, m1#{b}]; [r2.r3.f4, m2#{z}]";
On schema side, s1, s2....s6 are simple data type. m2:map(map(int)): meaning
m2 is a map of map. m2's value is a map and this inner map's value is a int
type. (key is always string type)
r2:record(r3:record(f3:float, f4)): meaning r2 is a record with one field
which is a record (r3). r3 is a record with two fields: f3: float and f4:
default type bytes.
On storage side, i.e [m1#{a}] meaning map m1 with key 'a' in this column
group. [s5, s6, m2#{x|y}] meaning s5, s6 and map m2 with key 'x' or 'y' in
this column group. [r2.r3.f4, m2#{z}] meaning record r2's record r3 with field
f4 and map m2 2ith key 'z' in this column group.
> Storage access layer
> --------------------
>
> Key: PIG-833
> URL: https://issues.apache.org/jira/browse/PIG-833
> Project: Pig
> Issue Type: New Feature
> Reporter: Jay Tang
> Attachments: hadoop20.jar.bz2, PIG-833-zebra.patch,
> PIG-833-zebra.patch.bz2, PIG-833-zebra.patch.bz2,
> TEST-org.apache.hadoop.zebra.pig.TestCheckin1.txt, test.out, zebra-javadoc.tgz
>
>
> A layer is needed to provide a high level data access abstraction and a
> tabular view of data in Hadoop, and could free Pig users from implementing
> their own data storage/retrieval code. This layer should also include a
> columnar storage format in order to provide fast data projection,
> CPU/space-efficient data serialization, and a schema language to manage
> physical storage metadata. Eventually it could also support predicate
> pushdown for further performance improvement. Initially, this layer could be
> a contrib project in Pig and become a hadoop subproject later on.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
