[
https://issues.apache.org/jira/browse/HIVE-21776?focusedWorklogId=247877&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-247877
]
ASF GitHub Bot logged work on HIVE-21776:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 24/May/19 04:52
Start Date: 24/May/19 04:52
Worklog Time Spent: 10m
Work Description: ashutosh-bapat commented on pull request #642:
HIVE-21776 : Replication fails to replicate a UDF with jar on HDFS during
incremental
URL: https://github.com/apache/hive/pull/642#discussion_r287214751
##########
File path:
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/ReplChangeManager.java
##########
@@ -356,6 +356,9 @@ public static FileInfo getFileInfo(Path src, String
checksumString, String srcCM
// Currently using fileuri#checksum#cmrooturi#subdirs as the format
public static String encodeFileUri(String fileUriStr, String fileChecksum,
String encodedSubDir)
throws IOException {
+ if (instance == null) {
+ throw new IOException("Uninitialized ReplChangeManager instance.");
+ }
String encodedUri = fileUriStr;
Review comment:
I agree this will be a problem in cloud to cloud replication, not just of
functions but also for other files.
When we create ReplCopyTask for FunctionHandler during load, we know that
this is going to a set of direct files. I think a better fix there would be to
tell ReplCopyTask that this is a direct set of files (instead of _files)
through a flag in constructor instead of relying upon annotations in URL. We
may want to change the structure of URL (jar url) later or interpret it
differently. This will help in cloud to cloud replication, which may have
require different structure of URL.
Given that there are other problems that we have to fix for cloud to cloud
other than this and also the uncertainty about what exactly will work with
cloud, we may adopt a different strategy here. So for the time being I will
leave this untouched.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 247877)
Time Spent: 1h (was: 50m)
> Replication fails to replicate a UDF with jar on HDFS during incremental
> ------------------------------------------------------------------------
>
> Key: HIVE-21776
> URL: https://issues.apache.org/jira/browse/HIVE-21776
> Project: Hive
> Issue Type: Bug
> Affects Versions: 4.0.0
> Reporter: Ashutosh Bapat
> Assignee: Ashutosh Bapat
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21776.01.patch, HIVE-21776.02.patch,
> HIVE-21776.03.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> TestReplicationScenariosAcrossInstances has test to test bootstrap of a UDF
> with jar on HDFS but no test for incremental. Add the same.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
