[
https://issues.apache.org/jira/browse/PIG-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anitha Raju updated PIG-2387:
-----------------------------
Description:
Hi,
When an input file of size greater than default split size is loaded using
BinStorage() and some processing is done, the task returns negative progress
Script
{code}
A = load 'input' using BinStorage() as (a:chararray);
B = filter A by (a matches '.*blinds.*');
store B into 'op';
{code}
Looking at the code, BinStorage which uses BinStorageRecordReader, has
getProgress()
{code}
public float getProgress() {
if (start == end) {
return 0.0f;
} else {
return Math.min(1.0f, (pos - start) / (float)(end - start));
}
}
{code}
In BinStorageRecordReader, pos is always 0 and not getting updated at any point.
So when the input file of size greater than default split size is loaded and
processed, the getProgress() method returns negative value, thus showing
negative progress.
Regards,
Anitha
was:
Hi,
When an input file of size greater than default split size is loaded using
BinStorage() and some processing is done, the task returns negative progress
Script
<code>
A = load 'input' using BinStorage() as (a:chararray);
B = filter A by (a matches '.*blinds.*');
store B into 'op';
<code>
Looking at the code, BinStorage which uses BinStorageRecordReader, has
getProgress()
<code>
public float getProgress() {
if (start == end) {
return 0.0f;
} else {
return Math.min(1.0f, (pos - start) / (float)(end - start));
}
}
<code>
In BinStorageRecordReader, pos is always 0 and not getting updated at any point.
So when the input file of size greater than default split size is loaded and
processed, the getProgress() method returns negative value, thus showing
negative progress.
Regards,
Anitha
> BinStorageRecordReader causes negative progress
> -----------------------------------------------
>
> Key: PIG-2387
> URL: https://issues.apache.org/jira/browse/PIG-2387
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0, 0.9.0
> Reporter: Anitha Raju
>
> Hi,
> When an input file of size greater than default split size is loaded using
> BinStorage() and some processing is done, the task returns negative progress
> Script
> {code}
> A = load 'input' using BinStorage() as (a:chararray);
> B = filter A by (a matches '.*blinds.*');
> store B into 'op';
> {code}
> Looking at the code, BinStorage which uses BinStorageRecordReader, has
> getProgress()
> {code}
> public float getProgress() {
> if (start == end) {
> return 0.0f;
> } else {
> return Math.min(1.0f, (pos - start) / (float)(end - start));
> }
> }
> {code}
> In BinStorageRecordReader, pos is always 0 and not getting updated at any
> point.
> So when the input file of size greater than default split size is loaded and
> processed, the getProgress() method returns negative value, thus showing
> negative progress.
> Regards,
> Anitha
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
