[
https://issues.apache.org/jira/browse/PIG-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nezih Yigitbasi updated PIG-3623:
---------------------------------
Attachment: PIG-3623.2.patch
This patch fixes the backward compatibility issues I found during my tests. The
solution is to use OptionBuilder to create options that take optional arguments
since addOption doesn't let me do that. And if we don't make loadKey and noWal
arguments optional (that is if we make 2nd argument of addOption true) then old
code that doesn't specify true|false breaks. Rohini, can you please review?
Btw I have run my own unit tests and couldn't get TestHBaseStorage running. I
get exceptions like :
[junit] 533071 [Thread-4871] WARN org.apache.hadoop.mapred.TaskTracker -
Error initializing att
empt_20140131132522179_0022_m_000001_1:
[junit] java.io.FileNotFoundException: File does not exist:
hdfs://localhost:44996/tmp/hadoop-nyi
gitba/mapred/system/job_20140131132522179_0022/jobToken
[junit] at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.j
ava:517)
[junit] at
org.apache.hadoop.mapred.TaskTracker.localizeJobTokenFile(TaskTracker.java:4250)
[junit] at
org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1158)
[junit] at
org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1099)
[junit] at
org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2382)
[junit] at java.lang.Thread.run(Thread.java:722)
[junit]
[junit] 533071 [Thread-4871] ERROR org.apache.hadoop.mapred.TaskStatus -
Trying to set finish ti
me for task attempt_20140131132522179_0022_m_000001_1 when no start time is
set, stackTrace is : java
.lang.Exception
[junit] at
org.apache.hadoop.mapred.TaskStatus.setFinishTime(TaskStatus.java:145)
[junit] at
org.apache.hadoop.mapred.TaskTracker$TaskInProgress.kill(TaskTracker.java:3094)
[junit] at
org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2392)
[junit] at java.lang.Thread.run(Thread.java:722)
[junit]
> HBaseStorage: setting loadKey and noWAL to false doesn't have any affect
> ------------------------------------------------------------------------
>
> Key: PIG-3623
> URL: https://issues.apache.org/jira/browse/PIG-3623
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.12.0
> Reporter: Michael Stefaniak
> Assignee: Nezih Yigitbasi
> Attachments: PIG-3623.1.patch, PIG-3623.2.patch, PIG-3623.patch
>
>
> The documentation for HBaseStorage
> (http://pig.apache.org/docs/r0.12.0/func.html#HBaseStorage)
> says -loadKey=(true|false) Load the row key as the first value in every tuple
> returned from HBase (default=false)
> However, looking at the source
> (http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/hbase/HBaseStorage.java)
> it is just doing a check for the existence of this option
> loadRowKey_ = configuredOptions_.hasOption("loadKey");
> So setting -loadKey=false in the options string, still results in a true value
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
