Hello, see : http://stackoverflow.com/questions/9665877/when-creating-and-loading-hfile-programmatically-to-hbase-new-entries-are-unavai
slim Tebourbi. Le 12 mars 2012 12:12, Nicolas Thiébaud <[email protected]> a écrit : > I'm trying to create HFiles programmatically and loading them in a running > HBase instance. I found a lot of info in HFileOutputFormat and in > LoadIncrementalHFiles > > I managed to create the new HFile, send it to the cluster. In the cluster > web interface the new store file appears but the new keyrange is > unavailable for query. > > InputStream stream = > ProgrammaticHFileGeneration.class.getResourceAsStream("ga-hourly.txt"); > BufferedReader reader = new BufferedReader(new InputStreamReader(stream)); > String line = null; > Map<byte[], String> rowValues = new HashMap<byte[], String>(); > while((line = reader.readLine())!=null) { > String[] vals = line.split(","); > String row = new > > StringBuilder(vals[0]).append(".").append(vals[1]).append(".").append(vals[2]).append(".").append(vals[3]).toString(); > rowValues.put(row.getBytes(), line); > } > List<byte[]> keys = new ArrayList<byte[]>(rowValues.keySet()); > Collections.sort(keys, byteArrComparator); > HBaseTestingUtility testingUtility = new HBaseTestingUtility(); > testingUtility.startMiniCluster(); > testingUtility.createTable("table".getBytes(), "data".getBytes()); > Writer writer = new HFile.Writer(testingUtility.getTestFileSystem(), > new Path("/tmp/hfiles/data/hfile"), > HFile.DEFAULT_BLOCKSIZE, Compression.Algorithm.NONE, > KeyValue.KEY_COMPARATOR); > for(byte[] key:keys) { > writer.append(new KeyValue(key, "data".getBytes(), "d".getBytes(), > rowValues.get(key).getBytes())); > } > writer.appendFileInfo(StoreFile.BULKLOAD_TIME_KEY, > Bytes.toBytes(System.currentTimeMillis())); > writer.appendFileInfo(StoreFile.MAJOR_COMPACTION_KEY, Bytes.toBytes(true)); > writer.close(); > Configuration conf = testingUtility.getConfiguration(); > LoadIncrementalHFiles loadTool = new LoadIncrementalHFiles(conf); > HTable hTable = new HTable(conf, "table".getBytes()); > loadTool.doBulkLoad(new Path("/tmp/hfiles"), hTable); > ResultScanner scanner = hTable.getScanner("data".getBytes()); > Result next = null; > System.out.println("Scanning"); > while((next = scanner.next()) != null) { > System.out.format("%s %s\n", new String(next.getRow()), new > String(next.getValue("data".getBytes(), "d".getBytes()))); > } > > What am I doing wrong ? I managed running M/R to Hbase, but I really want > the programmatic use case working.. >
