Thomas, When we try excluding the transitive dependencies using 'exclusion *', getting the below error while running maven build:
[ERROR] COMPILATION ERROR : [INFO] ------------------------------------------------------------- [ERROR] /C:/Work/Datatorrent-projects/bdp-datatorrent-mapr-streams/src/main/java/com/example/datatorrent/HbaseTableUpdate.java:[51,18] cannot access org.apache.hadoop.hbase.CellScannable class file for org.apache.hadoop.hbase.CellScannable not found [ERROR] /C:/Work/Datatorrent-projects/bdp-datatorrent-mapr-streams/src/main/java/com/example/datatorrent/HbaseTableUpdate.java:[105,10] cannot access org.apache.hadoop.hbase.io.HeapSize class file for org.apache.hadoop.hbase.io.HeapSize not found Thanks!! On Thu, Oct 20, 2016 at 1:44 PM, Thomas Weise <[email protected]> wrote: > This may also help: > > http://docs.datatorrent.com/troubleshooting/#hadoop-dependencies-conflicts > > > On Thu, Oct 20, 2016 at 11:39 AM, Thomas Weise <[email protected]> wrote: > >> Please see the HBase dependency and its exclusions here: >> >> https://github.com/apache/apex-malhar/blob/master/contrib/pom.xml#L342 >> >> Thanks, >> Thomas >> >> On Thu, Oct 20, 2016 at 9:07 AM, Jaspal Singh <[email protected] >> > wrote: >> >>> Team, >>> >>> While using the Hbase operator with Datatorrent application, we have >>> added hbase client dependency in pom.xml. Do we need to exclude transitive >>> hadoop dependencies using 'exclusion *' ? >>> >>> If we do that then HbaseConfiguration and Bytes methods are also getting >>> excluded and giving error in the application. Is there a way to fix it ?? >>> >>> >>> Thanks!! >>> >>> On Thu, Oct 20, 2016 at 1:38 AM, Tushar Gosavi <[email protected]> >>> wrote: >>> >>>> Hi Jaspal, >>>> >>>> You can pass the store name through property file, like >>>> >>>> <property> >>>> <name>dt.operator.HbaseOperatorName.store.tableName</name> >>>> <value>{name of the table}</value> >>>> </property> >>>> >>>> In the code, you can set the table name and other properties in >>>> constructor. >>>> {code} >>>> public static class Status2Hbase extends AbstractHBasePutOutputOperator >>>> <Status> >>>> { >>>> >>>> public Status2Hbase() >>>> { >>>> super(); >>>> // store is initialized to HBaseStore >>>> store.setTableName("nameofTable"); >>>> } >>>> >>>> @Override >>>> public Put operationPut(Status t) >>>> { >>>> Put put = new Put(ByteBuffer.allocate(8).put >>>> Long(t.getCreatedAt().getTime()).array()); >>>> put.add("cf".getBytes(), "text".getBytes(), t.getText().getBytes()); >>>> put.add("cf".getBytes(), "userid".getBytes(), t.getText().getBytes()); >>>> return put; >>>> } >>>> } >>>> {code} >>>> >>>> - Tushar. >>>> >>>> >>>> On Thu, Oct 20, 2016 at 11:59 AM, Jaspal Singh >>>> <[email protected]> wrote: >>>> > Hi Thomas, Thanks for sharing this example code. >>>> > Still I couldn't see where the hbase tablename is configured, it >>>> says in >>>> > description that it can be configured. >>>> > >>>> > Can you please highlight where it is specified ? >>>> > >>>> > Thanks!! >>>> > >>>> > >>>> > On Wednesday, October 19, 2016, Thomas Weise <[email protected]> wrote: >>>> >> >>>> >> Here is an example that uses HBase that may be helpful: >>>> >> >>>> >> >>>> >> https://github.com/apache/apex-malhar/blob/master/demos/twit >>>> ter/src/main/java/com/datatorrent/demos/twitter/TwitterDumpH >>>> BaseApplication.java >>>> >> >>>> >> Thomas >>>> >> >>>> >> On Wed, Oct 19, 2016 at 6:36 PM, Jaspal Singh < >>>> [email protected]> >>>> >> wrote: >>>> >>> >>>> >>> Where I need to set the table name. In property file or the >>>> application >>>> >>> code ? >>>> >>> >>>> >>> >>>> >>> Thanks!! >>>> >>> >>>> >>> >>>> >>> On Wednesday, October 19, 2016, Sanjay Pujare < >>>> [email protected]> >>>> >>> wrote: >>>> >>>> >>>> >>>> Take a look at >>>> >>>> https://github.com/apache/apex-malhar/tree/master/contrib/sr >>>> c/main/java/com/datatorrent/contrib/hbase >>>> >>>> . There are multiple output operators there. >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> You specify the table name using HBaseStore.setTableName >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> From: "Bandaru, Srinivas" <[email protected]> >>>> >>>> Reply-To: <[email protected]> >>>> >>>> Date: Wednesday, October 19, 2016 at 3:09 PM >>>> >>>> To: "[email protected]" <[email protected]> >>>> >>>> Subject: Datatorrent operator for Hbase >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> Hi, >>>> >>>> >>>> >>>> I want to write the data from an operator to a hbase table. Which >>>> >>>> operator I can use to write to Hbase table? >>>> >>>> >>>> >>>> Also how to specify the Hbase table name? >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> Thanks, >>>> >>>> >>>> >>>> Srinivas Bandaru >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> This e-mail, including attachments, may include confidential and/or >>>> >>>> proprietary information, and may be used only by the person or >>>> entity >>>> >>>> to which it is addressed. If the reader of this e-mail is not the >>>> >>>> intended >>>> >>>> recipient or his or her authorized agent, the reader is hereby >>>> notified >>>> >>>> that any dissemination, distribution or copying of this e-mail is >>>> >>>> prohibited. If you have received this e-mail in error, please >>>> notify the >>>> >>>> sender by replying to this message and delete this e-mail >>>> immediately. >>>> >> >>>> >> >>>> > >>>> >>> >>> >> >
