[jira] [Commented] (HBASE-5440) Allow Import to optionally use HFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13239861#comment-13239861 ] stack commented on HBASE-5440: -- @Lars Up to Ram. I've moved on. Allow Import to optionally use HFileOutputFormat Key: HBASE-5440 URL: https://issues.apache.org/jira/browse/HBASE-5440 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Fix For: 0.94.0 Attachments: 5440-v2.txt, 5440.txt importtsv support importing into a life table or to generate HFiles for bulk load. import should allow the same. Could even consider merging these tools into one (in principle the only difference is the parsing part - although that is maybe for a different jira). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5440) Allow Import to optionally use HFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13234568#comment-13234568 ] paul mackles commented on HBASE-5440: - Thanks Lars and Stack. I actually had a chance to play around with this a bit over the weekend and it certainly suited my purposes of being able to restore in a reasonable timeframe should disaster strike. We are actually still on 0.90.4 so I backported the relevant portions of the changes to that version of Import. Happy to create a patch if folks think that might be interesting. Allow Import to optionally use HFileOutputFormat Key: HBASE-5440 URL: https://issues.apache.org/jira/browse/HBASE-5440 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Fix For: 0.94.0 Attachments: 5440-v2.txt, 5440.txt importtsv support importing into a life table or to generate HFiles for bulk load. import should allow the same. Could even consider merging these tools into one (in principle the only difference is the parsing part - although that is maybe for a different jira). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5440) Allow Import to optionally use HFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13234595#comment-13234595 ] Lars Hofhansl commented on HBASE-5440: -- Hey Paul, I am glad this is useful for you. Reducing the timeframe for recovery is exactly what I had in mind with this. @Stack and @Ram: Are we doing more 0.90 releases? Should we add this? Allow Import to optionally use HFileOutputFormat Key: HBASE-5440 URL: https://issues.apache.org/jira/browse/HBASE-5440 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Fix For: 0.94.0 Attachments: 5440-v2.txt, 5440.txt importtsv support importing into a life table or to generate HFiles for bulk load. import should allow the same. Could even consider merging these tools into one (in principle the only difference is the parsing part - although that is maybe for a different jira). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5440) Allow Import to optionally use HFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13216343#comment-13216343 ] Hudson commented on HBASE-5440: --- Integrated in HBase-TRUNK #2669 (See [https://builds.apache.org/job/HBase-TRUNK/2669/]) HBASE-5440 Allow Import to optionally use HFileOutputFormat (Revision 1293101) Result = SUCCESS larsh : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java Allow Import to optionally use HFileOutputFormat Key: HBASE-5440 URL: https://issues.apache.org/jira/browse/HBASE-5440 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Fix For: 0.94.0 Attachments: 5440-v2.txt, 5440.txt importtsv support importing into a life table or to generate HFiles for bulk load. import should allow the same. Could even consider merging these tools into one (in principle the only difference is the parsing part - although that is maybe for a different jira). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5440) Allow import to optionally use HFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215071#comment-13215071 ] Hadoop QA commented on HBASE-5440: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12515813/5440.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -136 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 152 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestAtomicOperation org.apache.hadoop.hbase.coprocessor.TestClassLoading org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1027//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1027//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1027//console This message is automatically generated. Allow import to optionally use HFileOutputFormat Key: HBASE-5440 URL: https://issues.apache.org/jira/browse/HBASE-5440 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Fix For: 0.94.0 Attachments: 5440.txt importtsv support importing into a life table or to generate HFiles for bulk load. import should allow the same. Could even consider merging these tools into one (in principle the only difference is the parsing part - although that is maybe for a different jira). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5440) Allow import to optionally use HFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215178#comment-13215178 ] Lars Hofhansl commented on HBASE-5440: -- Ran the failed tests locally. They all pass. Allow import to optionally use HFileOutputFormat Key: HBASE-5440 URL: https://issues.apache.org/jira/browse/HBASE-5440 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Fix For: 0.94.0 Attachments: 5440.txt importtsv support importing into a life table or to generate HFiles for bulk load. import should allow the same. Could even consider merging these tools into one (in principle the only difference is the parsing part - although that is maybe for a different jira). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5440) Allow import to optionally use HFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215380#comment-13215380 ] Lars Hofhansl commented on HBASE-5440: -- Would review board help? This is actually a pretty simple change: import can optionally import into HFiles. In that case a new mapper and an additional reducer are used (similar to what importtsv does). Most of the changes are just so that code can be shared between KeyValueImporter and the existing Importer mapper classes. Allow import to optionally use HFileOutputFormat Key: HBASE-5440 URL: https://issues.apache.org/jira/browse/HBASE-5440 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Fix For: 0.94.0 Attachments: 5440.txt importtsv support importing into a life table or to generate HFiles for bulk load. import should allow the same. Could even consider merging these tools into one (in principle the only difference is the parsing part - although that is maybe for a different jira). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5440) Allow import to optionally use HFileOutputFormat
[
https://issues.apache.org/jira/browse/HBASE-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215388#comment-13215388
]
stack commented on HBASE-5440:
--
LGTM. Whats missing is better documentation in the usage for Import. This new
option will be under a rock unless its better surfaced. +1 on commit after
beefing up usage. Add some lines under here:
{code}
-System.err.println(Usage: Import tablename inputdir);
+System.err.println(Usage: Import [-D + BULK_OUTPUT_CONF_KEY
++ =/path/for/output] tablename inputdir);
{code}
... going on about what the -D thingy does.
Good stuff.
Allow import to optionally use HFileOutputFormat
Key: HBASE-5440
URL: https://issues.apache.org/jira/browse/HBASE-5440
Project: HBase
Issue Type: Improvement
Components: mapreduce
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
Fix For: 0.94.0
Attachments: 5440.txt
importtsv support importing into a life table or to generate HFiles for bulk
load.
import should allow the same.
Could even consider merging these tools into one (in principle the only
difference is the parsing part - although that is maybe for a different jira).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5440) Allow import to optionally use HFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215395#comment-13215395 ] Lars Hofhansl commented on HBASE-5440: -- Yeah, you're right of course :) Will do and a post a new patch soon. Allow import to optionally use HFileOutputFormat Key: HBASE-5440 URL: https://issues.apache.org/jira/browse/HBASE-5440 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Fix For: 0.94.0 Attachments: 5440.txt importtsv support importing into a life table or to generate HFiles for bulk load. import should allow the same. Could even consider merging these tools into one (in principle the only difference is the parsing part - although that is maybe for a different jira). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
