[
https://issues.apache.org/jira/browse/PHOENIX-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14646626#comment-14646626
]
James Taylor edited comment on PHOENIX-2154 at 7/29/15 7:19 PM:
----------------------------------------------------------------
Comment from [~maghamravikiran]:
Right now, the default reducer we use is KeyValueSortReducer . This aggregates
the mapper output and write as HFiles into a directory that the user has passed
as an argument to the job. After the map reduce job is complete, the
LoadIncrementalHFiles class does the task to move the HFiles onto the HBase
table.
Couple of possibilities that can happen in the above workflow
a) If all mappers succeed:
A happy path. HFiles get loaded into the HTable and we then update the
state to ACTIVE.
b) If few mappers fail:
The job is marked as failure. There can be instances that few mappers have
succeeded but since few(one or many) have failed, the job is marked as a
Failure. In this case, the output that was produced by the successful mappers
is discarded as part of the cleanup process in MapReduce. In this case, the
entire job has to be re-run.
It'd be good to make the mapper task be a kind of independent chunk of work,
which once complete, would not need to be run again in case the job is started
again.
Are you referring to the second case above? If so, I am not sure of an easy
way to have the output of mappers persist beyond the job. We can address this
by pushing the output KeyValue pairs onto a persistent store like HDFS rather
than writing to local filesystem directories. However, I see a challenge when
the job is run again. The case here is we have few records(in no sequence
order in the primary table) already processed and written to a persistent store
and if we do not want them to be processed again in the mapper, there should be
some flag or signal to avoid reading it. Mappers usually have region
boundaries, if when the job is run again, if we have the same region
boundaries, then we could definitely have a way to hack into the framework by
saying to run the mappers on only those regions where the earlier mappers
failed.
was (Author: jamestaylor):
Right now, the default reducer we use is KeyValueSortReducer . This aggregates
the mapper output and write as HFiles into a directory that the user has passed
as an argument to the job. After the map reduce job is complete, the
LoadIncrementalHFiles class does the task to move the HFiles onto the HBase
table.
Couple of possibilities that can happen in the above workflow
a) If all mappers succeed:
A happy path. HFiles get loaded into the HTable and we then update the
state to ACTIVE.
b) If few mappers fail:
The job is marked as failure. There can be instances that few mappers have
succeeded but since few(one or many) have failed, the job is marked as a
Failure. In this case, the output that was produced by the successful mappers
is discarded as part of the cleanup process in MapReduce. In this case, the
entire job has to be re-run.
It'd be good to make the mapper task be a kind of independent chunk of work,
which once complete, would not need to be run again in case the job is started
again.
Are you referring to the second case above? If so, I am not sure of an easy
way to have the output of mappers persist beyond the job. We can address this
by pushing the output KeyValue pairs onto a persistent store like HDFS rather
than writing to local filesystem directories. However, I see a challenge when
the job is run again. The case here is we have few records(in no sequence
order in the primary table) already processed and written to a persistent store
and if we do not want them to be processed again in the mapper, there should be
some flag or signal to avoid reading it. Mappers usually have region
boundaries, if when the job is run again, if we have the same region
boundaries, then we could definitely have a way to hack into the framework by
saying to run the mappers on only those regions where the earlier mappers
failed.
> Failure of one mapper should not affect other mappers in MR index build
> -----------------------------------------------------------------------
>
> Key: PHOENIX-2154
> URL: https://issues.apache.org/jira/browse/PHOENIX-2154
> Project: Phoenix
> Issue Type: Bug
> Reporter: James Taylor
>
> Once a mapper in the MR index job succeeds, it should not need to be re-done
> in the event of the failure of one of the other mappers. The initial
> population of an index is based on a snapshot in time, so new rows getting
> *after* the index build has started and/or failed do not impact it.
> Also, there's a 1:1 correspondence between index rows and table rows, so
> there's really no need to dedup. However, the index rows will have a
> different row key than the data table, so I'm not sure how the HFiles are
> split. Will they potentially overlap and is this an issue?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
