GitHub user GerardDellemann opened a pull request:
https://github.com/apache/metamodel/pull/182
Add Create, Update, Delete functionality for HBase
Create: It's possible to create tables and insert rows.
Update: Insert and update are the same in HBase.
Delete: It's possible to drop a table and delete rows based on the rowKey.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/GerardDellemann/metamodel
feature/hbase-improvements
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metamodel/pull/182.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #182
----
commit 6ad602bb9af70ed7214b6d7e4f00cd1d9a037f71
Author: HUMANINFERENCE\g.dellemann <g.dellemann@...>
Date: 2018-04-20T14:36:27Z
Notice with the builders that I tried to follow the setup used in other
projects, but this often didn't work for HBase's column-families.
The FindQualifiersDriver works, but only from command line within a VM
which runs Hadoop + HBase. It shows HBase working in combination with
MapReduce. The class will be improved so it's method can be called by other
classes.
All other classes have been tested by connecting it to our DataCleaner.
I'll make more Unit Tests next.
hbase/pom.xml changes: With 'hbase-client' I needed to remove 2 exclusion.
I've added 'hbase-server' dependency.
commit f15d02721910678e29c1452c1bf6f4b495609efc
Author: HUMANINFERENCE\g.dellemann <g.dellemann@...>
Date: 2018-04-23T14:53:03Z
Small improvements and fixed the existing Unit Test
commit 2008c7eb641c647ffe2588a1119b5d52f8fc9809
Author: arjansh <arjan.seijkens@...>
Date: 2018-04-30T09:49:52Z
Reverted changes to logic in core module and refactored HBaseColumn class
so it now extends the AbstractColumn class (and implement the Column class).
commit f2d2ff2a503ceebbd0bd3493d4df60cae08126ae
Author: Gerard Dellemann <gerard.dellemann@...>
Date: 2018-04-30T14:08:06Z
Merge pull request #1 from arjansh/feature/hbase-improvements-refactoring
Reverting some core changes and refactoring HBaseColumn class
commit 0ffeeb20af4d58061d042bb6ebccafb2ab7d6502
Author: Gerard Dellemann <g.dellemann@...>
Date: 2018-05-01T12:34:36Z
HBase improvements - Added row-deletion functionality, also made
HBaseDataContext implement UpdateableDataContext
commit 38d8bc3e18c6f966c2ef0fe661532d177e9359ee
Author: arjansh <arjan.seijkens@...>
Date: 2018-05-16T11:39:26Z
Made getColumnsInternal method synchronized to prevent
ConcurrentModificationExceptions when columns are retrieved from the Table in
an parallel manner.
commit 3ada7eb48e0599b507024b1b6c10be3b7cceb63f
Author: Gerard Dellemann <g.dellemann@...>
Date: 2018-05-24T14:12:20Z
HBase improvements: refactoring, new unittests
commit ea7356a4228f700b1b8f7115762cfc42ca6c92fc
Author: arjansh <arjan.seijkens@...>
Date: 2018-05-25T10:08:05Z
Refactored HBaseRowInsertionBuilder, so it uses the _columns of the
AbstractRowBuilder instead of its own, to make sure the model stays consistent.
commit 86084869ab7eefb92d983b472fb4664f3a0f742d
Author: arjansh <arjan.seijkens@...>
Date: 2018-05-25T11:12:36Z
Removed table exists check. Let's assume the createTable message throws an
exception when it fails to create the table.
commit 9ab30040e3311f5c9947615470ad1d366de92c0f
Author: Gerard Dellemann <g.dellemann@...>
Date: 2018-05-29T13:28:27Z
HBase Improvements - did some refactoring, added comments, added unittest
commit 54a900b4c9a608687239982d602c8eabc67f3afa
Author: Gerard Dellemann <g.dellemann@...>
Date: 2018-05-30T08:54:22Z
HBase unittests improvements (fixed issues when the testfile wasn't setup)
commit 5a28d73cdddfc8b11ca2478e390c8c94bcf34310
Author: Arjan Seijkens <a.seijkens@...>
Date: 2018-05-31T08:38:13Z
- Where missing added final qualifiers to method parameters.
- Moved static methods around to more logic locations. From my point of
view the HBaseColumn class is not a Helper class to provide those.
- Changed the signature of the HBaseRowInsertionBuilder constructor, so it
take a List of HBaseColumn object as a parameter, this makes it possible to
remove a few of the Helper methods which cast List<HBaseColumn> to List<Column>
and vice versa.
- Refactored the logic which looks up the id column in an array or list.
- Removed the "public" getHBaseColumnsInternal() method from HBaseTable,
which essentially was only invoked in combination with a static
getColumnFamilies method. Instead I added a getColumnFamilies() method to the
HBaseTable class which return the column families for that HBase table.
commit 703fb40e80a91521aca62fae918737aaa5e752bb
Author: Gerard Dellemann <gerard.dellemann@...>
Date: 2018-05-31T11:11:08Z
Merge pull request #3 from
GerardDellemann/feature/hbase-improvements-refactoring
Refactoring code and implementing feedback
commit cd755a1cf5ea3762463c577dd220162dee14dfd9
Author: Gerard Dellemann <g.dellemann@...>
Date: 2018-05-31T12:10:13Z
HBase improvements (small refactorings, add comments)
commit f4d2c97ad3a7fa41934251e6fd727639ed1bd300
Author: Gerard Dellemann <g.dellemann@...>
Date: 2018-05-31T14:30:20Z
HBase improvements (when creating a HBase table, the ID-column is not a
required parameter, because that's always created).
commit 717a3a443624f2da1a883a26d17a25886c77d67e
Author: Arjan Seijkens <a.seijkens@...>
Date: 2018-06-01T09:25:29Z
Refactored test so they no longer extends TestCase (and as such are no
longer JUnit 3, but JUnit 4 tests) and they use "assumeTrue" to check if they
should run or not. This reduces a lot of boiler plate code, because now we
don't need to call the "isConfigured" method in each test method.
Also added some extra null check in the code which cleans up after test,
because some nullpointer exceptions were thrown when the tests were skipped.
commit f48e9c833f574663a29078ba67b87f6932685287
Author: Gerard Dellemann <gerard.dellemann@...>
Date: 2018-06-03T08:44:17Z
Merge pull request #4 from
GerardDellemann/feature/hbase-improvements-assume-in-tests
Refactored to remove boiler plate isConfigured checks
commit 4a928d175897f6d71738a7113663723d160ffa0b
Author: Gerard Dellemann <g.dellemann@...>
Date: 2018-06-03T23:20:09Z
HBase improvements - Refactoring (mostly small improvements, the
getHBaseClient method has been moved to HBaseDataContent)
commit 5313a492f7c733be367c2738af14bc566a6090e1
Author: Arjan Seijkens <a.seijkens@...>
Date: 2018-06-06T08:12:02Z
HBaseClient.java:
- Remove overabundant `;` on line 111 ("try (final Table table =
_connection.getTable(TableName.valueOf(tableName));) {").
- Remove overabundant " == true" on line 112 ("if (rowExists(table,
rowKeyAsByteArray) == true) {").
- On line 115 use a Marker object when inserting variables into log
messages.
- Add final modifier on line 144 ("public void createTable(String
tableName, Set<String> columnFamilies) {").
HBaseCreateTableBuilder.java:
- On line 81 make the parameters of the setColumnFamilies method final.
HBaseRow.java:
- Revert remmoval of empty line (62).
HBaseRowDeletionBuilder.java:
- The _dataContext field member has to be made final
- On line 58 make the key par
----
---
