[
https://issues.apache.org/jira/browse/YARN-6094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15828401#comment-15828401
]
Sangjin Lee commented on YARN-6094:
-----------------------------------
bq. Yes. I wanted to load the coprocessor jar from classpath but I am not
certain how to do that in the unit test on the minicluster. We can discuss
further on this.
We spoke offline on this. There are more reliable ways to get to the right
enclosing jar using the {{Class}} and {{ClassLoader}}, but a larger issue is
that we cannot expect that the timeline service dependency is always in the
form of a jar file when these unit tests run. If we want to stick with a
dynamically loaded coprocessor, we need to create the jar on the fly.
Otherwise, it might be acceptable to use static loading for most of the unit
tests.
bq. Yes, dynamic loading API requires the coprocessor jar to be present on
HDFS. The Region Server does not need a restart when dynamic loading is used.
Thanks for the confirmation.
{quote}
These two classes DataGeneratorForTest and TestFlowDataGenerator are common to
most of the unit tests and they should have been put in a common package but
for some reason, we did not do that earlier. This move feels correct to me.
Regarding the diffs, I understand the changes are not easily understandable
since the files moved. I think I can upload a diff of the two files to make it
easier for review, let me know.
{quote}
I am +1 if the move makes sense. By the way, there is a way to detect renames
to generate a smaller patch. Add the renamed files (both deleted files and
added files) to git index or even commit it locally, and then use {{git diff
-M90%}} or even lower percentage to have git detect the rename.
> Update the coprocessor to be a dynamically loaded one
> -----------------------------------------------------
>
> Key: YARN-6094
> URL: https://issues.apache.org/jira/browse/YARN-6094
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Vrushali C
> Assignee: Vrushali C
> Labels: yarn-5355-merge-blocker
> Attachments: YARN-6094.001.patch, YARN-6094-YARN-5355.001.patch,
> YARN-6094-YARN-5355.002.patch
>
>
> The timeline service v2 code base on yarn now uses hbase 1.2.4 after
> YARN-5976.
> With this version of hbase, system classes (starting with org.apache.hadoop)
> can be loaded as table level coprocessors. Hence we should update the
> timeline service coprocessor to be a dynamically loaded one instead of static
> loading.
> This involves code changes as well as documentation updates for deployment.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
