read csv file as schema

Hi, Is there a way to load a csv file with header as schema? (the header's fields are the properties of the schema and the other list in the csv file will be in the schema format) For example: Namelast nameage Noamlavie 26 Map reduce

Re: read csv file as schema

In my company we had to write our own Loader/Storer UDFs for this. On Wed, Nov 27, 2013 at 6:00 PM, Noam Lavie noam.la...@pontis.com wrote: Hi, Is there a way to load a csv file with header as schema? (the header's fields are the properties of the schema and the other list in the csv file

Storing tuple into HBaseStorage

Hello Pig users, I want to store the entire tuple into hbase from Pig using HBaseStorage. I know that I can do something like: output = as (c1:bytearray, c2:bytearray, cN:bytearray); STORE output INTO 'hbase://outputtable' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('f1:c1

Re: Storing tuple into HBaseStorage

You should be able to use a Pig map to do this. Use the column name as the key in the map and the value as the value. You should be able to use the builtin TOMAP function to generate the map ( http://pig.apache.org/docs/r0.11.0/func.html#tomap). The HBaseStorage documentation gives an example

Re: Storing tuple into HBaseStorage

Hi Shawn, Thanks for the advice. Can TOMAP generate a map from tuple using the alias of the field in the tuple as the key of the map and the field value as the value of the map? Form the documentation, TOMAP syntax is: TOMAP(key-expression, value-expression [, key-expression, value-expression

Unable to run PIG local as regular user (non-root)

Hi, I've been following the book Hadoop Definitive Guide and a few other resources while exploring PIG. I am able to run as root, but not as a regular user. I am running in local mode. I've found a few links, but they haven't helped me solve my issue. The most relevant seems to be:

Pig syntax to access fields of records in an array

Hi, We have an Avro file of which a field that is an array of tuples as follows: cam:bag{ARRAY_ELEM:tuple(BIDCOUNT: int, ... I tried to access BIDCOUNT with 'cam.BIDCOUNT'. It is not working. Any body knows how to access BIDCOUNT? Thanks. Ey-Chih Chow

Re: Storing tuple into HBaseStorage

That is a very good question. I am not sure if there is an easy way to use the alias of the field as the key. I looked at the Tuple class definition ( http://pig.apache.org/docs/r0.9.1/api/org/apache/pig/data/Tuple.html) and it appears it does not give an option to get the name associated with a

Re: Using variables generated by FOREACH command

afaik, you can also do generate flatten(group) as (page_name,web_session_id), 2013/11/15 noor_cs...@yahoo.com Hey, Have you checked that you are really getting all the columns you have specified in x? can you tell me what dump x is giving you? When you flatten group in x, try

Re: Storing tuple into HBaseStorage

Hi Shawn, I see your point now. Thank you for your help! Jerry On Wed, Nov 27, 2013 at 6:14 PM, Shawn Hermans shawnherm...@gmail.comwrote: That is a very good question. I am not sure if there is an easy way to use the alias of the field as the key. I looked at the Tuple class definition (

Re: Need help

This question belongs on the user list. The dev list is meant for Pig developers to discuss issues related to the development of Pig. I’ve forwarded this to the user list. It also helps tremendously if you format your data and scripts nicely as they’re much easier to read and understand. I use a

Re: Multiple Input Schemas in AvroStorage() fails

I guess you need to specify 'multiple_schema' in AvroStorage On Thu, Nov 28, 2013 at 4:07 AM, Mangtani, Kushal kushal.mangt...@viasat.com wrote: Hi, I'm one of the several Pig Developer/User community.I have a question regarding Avro1.6.1 and Pig0.11 compatibility. In ref to