Ok, I'll create a Hive jira for it then. I can't find a Hadoop one for
adding custom line separators in the TextInputFormat either, if there is
none I'll create that too.

/Johan

Zheng Shao wrote:
> Yes this is a known issue. "lines terminated by" is not supported yet
> because the text input format do not allow configurable line
> separators yet.
> 
> 
> Zheng
> 
> 
> On 2/24/09, Johan Oskarsson <[email protected]> wrote:
>> I've been trying to use a text file with the field separator \001 and
>> line separator \002\n in Hive, similar to what's described here
>> http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL.
>>
>> I've set the "lines terminated by" to \002 but when selecting data the
>> last column still includes that character, if the last col is a string.
>> If it's an int the value fails to parse and is left as null.
>>
>> Is this a known issue? I can't find a ticket for it.
>> I assume the TextInputFormat takes care of the \n so that I only need to
>> use \002 as the termination.
>>
>> Example create queries
>>
>> create table artistsong (export_date bigint, artist_id int, song_id int)
>> ROW FORMAT DELIMITED FIELDS TERMINATED BY '\001' lines terminated by '\002'
>>
>> and this one:
>>
>> create table artist (export_date bigint, artist_id int, name string,
>> is_actual_artist int, view_url string) ROW FORMAT DELIMITED FIELDS
>> TERMINATED BY '\001' lines terminated by '\002'
>>
>> /Johan
>>
> 

Reply via email to