[jira] [Commented] (HBASE-14128) Fix inability to run Multiple MR over the same Snapshot
[
https://issues.apache.org/jira/browse/HBASE-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413567#comment-15413567
]
Hadoop QA commented on HBASE-14128:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s {color}
| {color:red} HBASE-14128 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12756303/HBASE-14128-v0.patch |
| JIRA Issue | HBASE-14128 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3026/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Fix inability to run Multiple MR over the same Snapshot
> ---
>
> Key: HBASE-14128
> URL: https://issues.apache.org/jira/browse/HBASE-14128
> Project: HBase
> Issue Type: Bug
> Components: mapreduce, snapshots
>Reporter: Matteo Bertozzi
>Assignee: santosh kumar
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-14128-v0.patch
>
>
> from the list, running multiple MR over the same snapshot does not work
> {code}
> public static void copySnapshotForScanner(Configuration conf, FileSystem ..
> RestoreSnapshotHelper helper = new RestoreSnapshotHelper(conf, fs,
> manifest, manifest.getTableDescriptor(), restoreDir, monitor, status);
> {code}
> the problem is that manifest.getTableDescriptor() will try to clone the
> snapshot with the same target name. ending up in "file already exist"
> exceptions.
> we just need to clone that descriptor and generate a new target table name
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14128) Fix inability to run Multiple MR over the same Snapshot
[
https://issues.apache.org/jira/browse/HBASE-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15089312#comment-15089312
]
Enis Soztutar commented on HBASE-14128:
---
This came up in the dev list again.
Skimmed the patch. Looks good to go. Matteo, is there anything else to do for
this. I can help if you don't have time.
> Fix inability to run Multiple MR over the same Snapshot
> ---
>
> Key: HBASE-14128
> URL: https://issues.apache.org/jira/browse/HBASE-14128
> Project: HBase
> Issue Type: Bug
> Components: mapreduce, snapshots
>Reporter: Matteo Bertozzi
>Assignee: santosh kumar
>Priority: Minor
> Labels: beginner, noob
> Attachments: HBASE-14128-v0.patch
>
>
> from the list, running multiple MR over the same snapshot does not work
> {code}
> public static void copySnapshotForScanner(Configuration conf, FileSystem ..
> RestoreSnapshotHelper helper = new RestoreSnapshotHelper(conf, fs,
> manifest, manifest.getTableDescriptor(), restoreDir, monitor, status);
> {code}
> the problem is that manifest.getTableDescriptor() will try to clone the
> snapshot with the same target name. ending up in "file already exist"
> exceptions.
> we just need to clone that descriptor and generate a new target table name
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14128) Fix inability to run Multiple MR over the same Snapshot
[
https://issues.apache.org/jira/browse/HBASE-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15089316#comment-15089316
]
Hadoop QA commented on HBASE-14128:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} HBASE-14128 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/latest/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12756303/HBASE-14128-v0.patch |
| JIRA Issue | HBASE-14128 |
| Powered by | Apache Yetus 0.1.0 http://yetus.apache.org |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/30/console |
This message was automatically generated.
> Fix inability to run Multiple MR over the same Snapshot
> ---
>
> Key: HBASE-14128
> URL: https://issues.apache.org/jira/browse/HBASE-14128
> Project: HBase
> Issue Type: Bug
> Components: mapreduce, snapshots
>Reporter: Matteo Bertozzi
>Assignee: santosh kumar
>Priority: Minor
> Labels: beginner, noob
> Attachments: HBASE-14128-v0.patch
>
>
> from the list, running multiple MR over the same snapshot does not work
> {code}
> public static void copySnapshotForScanner(Configuration conf, FileSystem ..
> RestoreSnapshotHelper helper = new RestoreSnapshotHelper(conf, fs,
> manifest, manifest.getTableDescriptor(), restoreDir, monitor, status);
> {code}
> the problem is that manifest.getTableDescriptor() will try to clone the
> snapshot with the same target name. ending up in "file already exist"
> exceptions.
> we just need to clone that descriptor and generate a new target table name
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14128) Fix inability to run Multiple MR over the same Snapshot
[
https://issues.apache.org/jira/browse/HBASE-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804470#comment-14804470
]
stack commented on HBASE-14128:
---
+1 (after chatting about patch offline).
Nits below. Fix the latter minor on commit.
Seems fine making this non-pubilc since class is private, evolving...
static SnapshotManifest getSnapshotManifest
It is opaque doing the below where a restoreId becomes a tablename... just have
the passed in arg be tablename?
public static List getSplits(Scan scan, SnapshotManifest manifest,
320 List regionManifests, Path restoreDir, String
restoreId, Configuration conf)
> Fix inability to run Multiple MR over the same Snapshot
> ---
>
> Key: HBASE-14128
> URL: https://issues.apache.org/jira/browse/HBASE-14128
> Project: HBase
> Issue Type: Bug
> Components: mapreduce, snapshots
>Reporter: Matteo Bertozzi
>Assignee: santosh kumar
>Priority: Minor
> Labels: beginner, noob
> Attachments: HBASE-14128-v0.patch
>
>
> from the list, running multiple MR over the same snapshot does not work
> {code}
> public static void copySnapshotForScanner(Configuration conf, FileSystem ..
> RestoreSnapshotHelper helper = new RestoreSnapshotHelper(conf, fs,
> manifest, manifest.getTableDescriptor(), restoreDir, monitor, status);
> {code}
> the problem is that manifest.getTableDescriptor() will try to clone the
> snapshot with the same target name. ending up in "file already exist"
> exceptions.
> we just need to clone that descriptor and generate a new target table name
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14128) Fix inability to run Multiple MR over the same Snapshot
[
https://issues.apache.org/jira/browse/HBASE-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791074#comment-14791074
]
Hadoop QA commented on HBASE-14128:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12756303/HBASE-14128-v0.patch
against master branch at commit bd26386dc7205c9b30b8488bc094bd380ec09adb.
ATTACHMENT ID: 12756303
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1837 checkstyle errors (more than the master's current 1835 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestImportExport
org.apache.hadoop.hbase.util.TestProcessBasedCluster
org.apache.hadoop.hbase.regionserver.TestWALLockup
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15625//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15625//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15625//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15625//console
This message is automatically generated.
> Fix inability to run Multiple MR over the same Snapshot
> ---
>
> Key: HBASE-14128
> URL: https://issues.apache.org/jira/browse/HBASE-14128
> Project: HBase
> Issue Type: Bug
> Components: mapreduce, snapshots
>Reporter: Matteo Bertozzi
>Assignee: santosh kumar
>Priority: Minor
> Labels: beginner, noob
> Attachments: HBASE-14128-v0.patch
>
>
> from the list, running multiple MR over the same snapshot does not work
> {code}
> public static void copySnapshotForScanner(Configuration conf, FileSystem ..
> RestoreSnapshotHelper helper = new RestoreSnapshotHelper(conf, fs,
> manifest, manifest.getTableDescriptor(), restoreDir, monitor, status);
> {code}
> the problem is that manifest.getTableDescriptor() will try to clone the
> snapshot with the same target name. ending up in "file already exist"
> exceptions.
> we just need to clone that descriptor and generate a new target table name
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14128) Fix inability to run Multiple MR over the same Snapshot
[
https://issues.apache.org/jira/browse/HBASE-14128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643306#comment-14643306
]
santosh kumar commented on HBASE-14128:
---
Noob here - so could you look at my questions:
1. What version is this targeted for?
2. I looked at the code and find that this method is being used by
MultiTableSnapshotInputFormat and TableSnapshotInputFormat and for those 2 I
can understand the usage. However, ScanPerformanceEvaluation uses this as well,
but it is a Test class(being found in src/test), for which I don't understand
the usage.
3. What would the target table name be based off ? A timestamp to prevent
duplicate?
4. Is the underlying assumption that the underlying Hbase table doesn't exist
when TableMapReduceUtil.initTableSnapshotMapperJob makes an attempt to run this
restore? Would it be better if the 'target' table is passed as a parameter
instead?
Fix inability to run Multiple MR over the same Snapshot
---
Key: HBASE-14128
URL: https://issues.apache.org/jira/browse/HBASE-14128
Project: HBase
Issue Type: Bug
Components: mapreduce, snapshots
Reporter: Matteo Bertozzi
Assignee: santosh kumar
Priority: Minor
Labels: beginner, noob
from the list, running multiple MR over the same snapshot does not work
{code}
public static void copySnapshotForScanner(Configuration conf, FileSystem ..
RestoreSnapshotHelper helper = new RestoreSnapshotHelper(conf, fs,
manifest, manifest.getTableDescriptor(), restoreDir, monitor, status);
{code}
the problem is that manifest.getTableDescriptor() will try to clone the
snapshot with the same target name. ending up in file already exist
exceptions.
we just need to clone that descriptor and generate a new target table name
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
