[
https://issues.apache.org/jira/browse/FALCON-1102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14507001#comment-14507001
]
Peeyush Bishnoi commented on FALCON-1102:
-----------------------------------------
Patch is attached that will populate the GraphDB with BYTESCOPIED once data
transfer completed from HDFS recipe. In the given patch, distcp job counter
"BYTESCOPIED" will be temporarily stored in the counter file inside the
workflow log directory once Replication workflow finished. After this,
FalconPostProcessing (invoked through different workflow) will make a check for
the counter file to read the counter information. Once counter information is
available, it will be added to WorkflowExecutionContext class.
On JMS invocation from FalconPostProcessing class, value of "BYTESCOPIED"
counter (stored in WorkflowExecutionContext) will be accessed in
InstanceRelationshipGraphBuilder class and added to respective process instance
vertex in GraphDB so that UI can access. Corresponding test cases
"testLineageForDRCounter" has been added for this in MetadataMappingServiceTest
class.
Please review.
> Gather data transfer detail of replication job submitted from HDFS recipe
> -------------------------------------------------------------------------
>
> Key: FALCON-1102
> URL: https://issues.apache.org/jira/browse/FALCON-1102
> Project: Falcon
> Issue Type: Bug
> Affects Versions: 0.7
> Reporter: Peeyush Bishnoi
> Assignee: Peeyush Bishnoi
> Fix For: 0.7
>
> Attachments: FALCON-1102.patch
>
>
> Falcon UI has a requirement to show data transfer details from replication
> job invoked through HDFS recipe. To carry out this, we need to capture the
> bytes transferred from source to destination of replication job and then
> populate to backend store from where UI can access.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
