Hi Ted,
thanks for your quick answer. It would indeed. But since we have in
the vicinity of 6K such log-lines for the table in a single
regionserver, should I then interpret the the number coming after the
RS_OPEN_REGION-{servername:port}- as the thread id? E.g.
RS_OPEN_REGION-sild03:16020-16 as thread number 16 that operates on
the region?
Best regards,
Steen
On Tue, Sep 12, 2017 at 12:48 PM, Ted Yu <[email protected]> wrote:
> loadTableCoprocessors() is called from RegionCoprocessorHost ctor.
> In HRegion ctor, you would find:
>
> this.coprocessorHost = new RegionCoprocessorHost(this, rsServices,
> conf);
>
> Meaning RegionCoprocessorHost is instantiated per region.
> This can explain why you saw the log multiple times.
>
> On Tue, Sep 12, 2017 at 2:34 AM, Steen Manniche <[email protected]> wrote:
>
>> Hi list.
>>
>> I have a coprocessor which has been loaded on a table 'plot'. The
>> table has 2697 regions across 8 nodes. Recently, I have observed some
>> strangeness in the log-files:
>>
>> 2017-09-12 11:00:28,461 INFO [RS_OPEN_REGION-sild03:16020-16]
>> regionserver.RegionCoprocessorHost: Loaded coprocessor
>> net.nzcorp.hbase.tableevent_signaler.TableEventSignaler from HTD of
>> protein_ee successfully.
>> 2017-09-12 11:00:29,080 INFO [RS_OPEN_REGION-sild03:16020-16]
>> regionserver.RegionCoprocessorHost: Loaded coprocessor
>> net.nzcorp.hbase.tableevent_signaler.TableEventSignaler from HTD of
>> protein_ee successfully.
>> 2017-09-12 11:00:29,648 INFO [RS_OPEN_REGION-sild03:16020-16]
>> regionserver.RegionCoprocessorHost: Loaded coprocessor
>> net.nzcorp.hbase.tableevent_signaler.TableEventSignaler from HTD of
>> protein_ee successfully.
>> 2017-09-12 11:00:30,464 INFO [RS_OPEN_REGION-sild03:16020-16]
>> regionserver.RegionCoprocessorHost: Loaded coprocessor
>> net.nzcorp.hbase.tableevent_signaler.TableEventSignaler from HTD of
>> protein_ee successfully.
>> 2017-09-12 11:00:31,204 INFO [RS_OPEN_REGION-sild03:16020-16]
>> regionserver.RegionCoprocessorHost: Loaded coprocessor
>> net.nzcorp.hbase.tableevent_signaler.TableEventSignaler from HTD of
>> protein_ee successfully.
>> 2017-09-12 11:00:31,874 INFO [RS_OPEN_REGION-sild03:16020-16]
>> regionserver.RegionCoprocessorHost: Loaded coprocessor
>> net.nzcorp.hbase.tableevent_signaler.TableEventSignaler from HTD of
>> protein_ee successfully.
>> 2017-09-12 11:00:32,253 INFO [RS_OPEN_REGION-sild03:16020-16]
>> regionserver.RegionCoprocessorHost: Loaded coprocessor
>> net.nzcorp.hbase.tableevent_signaler.TableEventSignaler from HTD of
>> protein_ee successfully.
>> 2017-09-12 11:01:52,703 INFO [RS_OPEN_REGION-sild03:16020-16]
>> regionserver.RegionCoprocessorHost: Loaded coprocessor
>> net.nzcorp.hbase.tableevent_signaler.TableEventSignaler from HTD of
>> protein_ee successfully.
>> 2017-09-12 11:02:08,403 INFO [RS_OPEN_REGION-sild03:16020-16]
>> regionserver.RegionCoprocessorHost: Loaded coprocessor
>> net.nzcorp.hbase.tableevent_signaler.TableEventSignaler from HTD of
>> protein_ee successfully.
>> 2017-09-12 11:02:18,215 INFO [RS_OPEN_REGION-sild03:16020-16]
>> regionserver.RegionCoprocessorHost: Loaded coprocessor
>> net.nzcorp.hbase.tableevent_signaler.TableEventSignaler from HTD of
>> protein_ee successfully.
>> 2017-09-12 11:02:37,779 INFO [RS_OPEN_REGION-sild03:16020-16]
>> regionserver.RegionCoprocessorHost: Loaded coprocessor
>> net.nzcorp.hbase.tableevent_signaler.TableEventSignaler from HTD of
>> protein_ee successfully.
>> 2017-09-12 11:02:49,227 INFO [RS_OPEN_REGION-sild03:16020-16]
>> regionserver.RegionCoprocessorHost: Loaded coprocessor
>> net.nzcorp.hbase.tableevent_signaler.TableEventSignaler from HTD of
>> protein_ee successfully.
>> 2017-09-12 11:03:01,271 INFO [RS_OPEN_REGION-sild03:16020-16]
>> regionserver.RegionCoprocessorHost: Loaded coprocessor
>> net.nzcorp.hbase.tableevent_signaler.TableEventSignaler from HTD of
>> protein_ee successfully.
>> 2017-09-12 11:09:10,639 INFO [RS_OPEN_REGION-sild03:16020-16]
>> regionserver.RegionCoprocessorHost: Loaded coprocessor
>> net.nzcorp.hbase.tableevent_signaler.TableEventSignaler from HTD of
>> protein_ee successfully.
>> 2017-09-12 11:14:08,939 INFO [RS_OPEN_REGION-sild03:16020-16]
>> regionserver.RegionCoprocessorHost: Loaded coprocessor
>> net.nzcorp.hbase.tableevent_signaler.TableEventSignaler from HTD of
>> protein_ee successfully.
>>
>> All in all, There are thousand of these log-lines on the regionserver
>> logs. The coprocessor is quite simple
>> (https://github.com/manniche/hbase-table-event-signaler/
>> blob/develop/src/main/java/net/nzcorp/hbase/tableevent_
>> signaler/TableEventSignaler.java)
>>
>> I would assume to find this line exactly once per region, but it seems
>> that the `start()` method on the coprocessor is called multiple times?
>>
>> the hbase describe table shows:
>>
>> hbase(main):007:0> describe 'plot'
>> Table plot is ENABLED
>> plot, {TABLE_ATTRIBUTES => {coprocessor$5 =>
>> 'hdfs:///user/hbase/nzcorp-tableevent-signaler-0.10.0.
>> jar|net.nzcorp.hbase.tableevent_signaler.TableEventSignaler|5|
>> destination_table=plot,secondary_index_table=plot_index,seco
>> ndary_index_cf=p,source_column_family=e,target_column_
>> family=e,amq_address=amqp://*******:******@rmq-server1:5672/
>> hbase_events,send_value=true,filter_qualifiers=dna_
>> accession_number|feature_accession_number'}
>> [... info on column families ...]
>> 9 row(s) in 0.3650 seconds
>>
>> We are running hbase version 1.1.2
>>
>> Best regards,
>> Steen
>>
