Re: Incrementally load HFiles outside of MR/Spark

2018-02-25 Thread Ted Yu
You can refer to HFilePerformanceEvaluation where creation of Writer is demonstrated: writer = HFile.getWriterFactoryNoCache(conf) .withPath(fs, mf) .withFileContext(hFileContext) .withComparator(CellComparator.getInstance()) .create(); Cheers

Incrementally load HFiles outside of MR/Spark

2018-02-25 Thread Mike Thomsen
I'm looking into creating HFiles directly from NiFi using the HBase API. It seems pretty straight forward: 1. Open a HFile.Writer pointing to a file path in HDFS. 2. Write the cells with the HFile API. 3. Call the incremental loader API to have it tell HBase to load the generated segments. Is