[
https://issues.apache.org/jira/browse/ACCUMULO-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13850524#comment-13850524
]
Alex Moundalexis commented on ACCUMULO-2014:
--------------------------------------------
Also possible all that was needed was a restart of the tablet server, that the
map size was incidental.
> Feedback loop on Tracer table given small in-memory map on server
> -----------------------------------------------------------------
>
> Key: ACCUMULO-2014
> URL: https://issues.apache.org/jira/browse/ACCUMULO-2014
> Project: Accumulo
> Issue Type: Bug
> Components: trace, tserver
> Affects Versions: 1.4.3
> Reporter: Sean Busbey
> Assignee: Eric Newton
> Priority: Minor
>
> Under the right conditions the tracer table can get into a feedback loop
> where messages about flushes to a tracer table tablet keep arriving and
> triggering subsequent flush operations.
> This was observed on a 1.4.3 cluster initially configured for 256MB in-memory
> maps.
> This manifests as a high rate of ingest to the tracer table, even in the
> absence of client activity or any client created tables. i.e. on a 3 worker
> node cluster with no external activity 500 entries/second into the tracer
> table.
> Viewing the tserver logs:
> {noformat}
> tserver.log (less AUDIT):
> $ tail -50 tserver-3-example.com.log | grep -v AUDIT
> 2013-12-10 19:08:22,433 INFO
> org.apache.accumulo.server.tabletserver.TabletServer: Adding 1 logs for
> extent 1<;b99f as alias 5
> 2013-12-10 19:08:22,627 INFO
> org.apache.accumulo.server.tabletserver.TabletServer: Adding 1 logs for
> extent !0<;~ as alias 2
> 2013-12-10 19:08:23,431 INFO
> org.apache.accumulo.server.tabletserver.TabletServer: Adding 1 logs for
> extent 1<;b99f as alias 5
> 2013-12-10 19:08:23,735 INFO
> org.apache.accumulo.server.tabletserver.TabletServer: Adding 1 logs for
> extent !0<;~ as alias 2
> 2013-12-10 19:08:24,431 INFO
> org.apache.accumulo.server.tabletserver.TabletServer: Adding 1 logs for
> extent 1<;b99f as alias 5
> 2013-12-10 19:08:24,875 INFO
> org.apache.accumulo.server.tabletserver.TabletServer: Adding 1 logs for
> extent !0<;~ as alias 2
> 2013-12-10 19:08:26,431 INFO
> org.apache.accumulo.server.tabletserver.TabletServer: Adding 1 logs for
> extent 1<;b99f as alias 5
> 2013-12-10 19:08:26,486 INFO
> org.apache.accumulo.server.tabletserver.TabletServer: Adding 1 logs for
> extent !0<;~ as alias 2
> tserver.log (full)
> 2013-12-10 19:07:44,479 AUDIT org.apache.accumulo.server.security.Auditor:
> Using credentials root: authenticated
> 2013-12-10 19:07:44,480 AUDIT org.apache.accumulo.server.security.Auditor:
> Using credentials !SYSTEM: checked permission WRITE on table 1 for root
> 2013-12-10 19:07:44,561 AUDIT org.apache.accumulo.server.security.Auditor:
> Using credentials !SYSTEM: checked permission WRITE on table !0 for !SYSTEM
> 2013-12-10 19:07:45,379 AUDIT org.apache.accumulo.server.security.Auditor:
> Using credentials !SYSTEM: checked permission READ on table !0 for !SYSTEM
> 2013-12-10 19:07:45,379 AUDIT org.apache.accumulo.server.security.Auditor:
> Using credentials !SYSTEM: got authorizations for !SYSTEM
> 2013-12-10 19:07:45,479 AUDIT org.apache.accumulo.server.security.Auditor:
> Using credentials root: authenticated
> 2013-12-10 19:07:45,480 AUDIT org.apache.accumulo.server.security.Auditor:
> Using credentials !SYSTEM: checked permission WRITE on table 1 for root
> 2013-12-10 19:07:45,481 INFO
> org.apache.accumulo.server.tabletserver.TabletServer: Adding 1 logs for
> extent 1<;b99f as alias 5
> 2013-12-10 19:07:45,554 AUDIT org.apache.accumulo.server.security.Auditor:
> Using credentials !SYSTEM: checked permission WRITE on table !0 for !SYSTEM
> 2013-12-10 19:07:45,555 INFO
> org.apache.accumulo.server.tabletserver.TabletServer: Adding 1 logs for
> extent !0<;~ as alias 2
> 2013-12-10 19:07:45,731 AUDIT org.apache.accumulo.server.security.Auditor:
> Using credentials !SYSTEM: checked permission WRITE on table !0 for !SYSTEM
> 2013-12-10 19:07:46,479 AUDIT org.apache.accumulo.server.security.Auditor:
> Using credentials root: authenticated
> 2013-12-10 19:07:46,481 AUDIT org.apache.accumulo.server.security.Auditor:
> Using credentials !SYSTEM: checked permission WRITE on table 1 for root
> 2013-12-10 19:07:46,482 INFO
> org.apache.accumulo.server.tabletserver.TabletServer: Adding 1 logs for
> extent 1<;b99f as alias 5
> 2013-12-10 19:07:46,574 AUDIT org.apache.accumulo.server.security.Auditor:
> Using credentials !SYSTEM: checked permission WRITE on table !0 for !SYSTEM
> 2013-12-10 19:07:46,575 INFO
> org.apache.accumulo.server.tabletserver.TabletServer: Adding 1 logs for
> extent !0<;~ as alias 2
> 2013-12-10 19:07:46,743 AUDIT org.apache.accumulo.server.security.Auditor:
> Using credentials !SYSTEM: checked permission WRITE on table !0 for !SYSTEM
> {noformat}
> note that this example relies on the default configuration, using "root" as
> the user for the tracer role. If your tracer user is set to something else,
> that is the user that would show up in the credential checks above.
> Current work around is to increase the memory map size.
> We should attempt to detect this situation and at least issue a clear log
> message. It would be nicer if we could detect it and turn off tracing
> messages related to the tracing table.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
