Re: [External] RE: Problem creating tables in Accumulo

2023-02-22 Thread Samudrala, Ranganath [USA] via user 
Thank you every one for the response.

I think the problem is related to slow / intermittent response from Amazon S3 
due the network situation we are in. I am trying to use an internal S3 bucket 
to see if the problem persists.

thanks again.
Ranga

From: dlmar...@comcast.net 
Sent: Wednesday, February 22, 2023 2:59 PM
To: user@accumulo.apache.org ; Samudrala, Ranganath 
[USA] 
Subject: [External] RE: Problem creating tables in Accumulo


It looks like the issue is that `maxTime` is not being set at [0], so `time` is 
not being set at [1]. If I had to guess, Accumulo is having an issue talking 
with ZooKeeper. Were there errors in the log when initializing Accumulo? Are 
you able to `exec` into one of the Accumulo Kubernetes pods and confirm 
connectivity with Accumulo?

[0] 
https://github.com/apache/accumulo/blob/1.10/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/TabletData.java#L159
[1] 
https://github.com/apache/accumulo/blob/1.10/server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/TabletData.java#L166



From: Samudrala, Ranganath [USA] via user 
Sent: Tuesday, February 21, 2023 1:43 PM
To: Samudrala, Ranganath [USA] via user 
Subject: Problem creating tables in Accumulo



Hello,



Accumulo is running in Kubernetes cluster. Metdata data is stored in HDFS and 
table data is stored in Amazon S3. No attempt has been made yet to store/insert 
table data. But, when I attempt to create a table in Accumulo via Java API, I 
see the following error.





WARN : org.apache.accumulo.core.clientImpl.ThriftTransportPool - Thread "main" 
stuck on IO to accumulo-masters-0.accumulo-masters.acc

umulo-k8s.svc.cluster.local: (0) for at least 120033 ms



I see the below error in T-SERVER logs:

2023-02-21 18:04:09,373 WARN [Root tablet assignment retry] 
[org.apache.accumulo.tserver.TabletServer]: exception trying to assign tablet 
+r<< 
hdfs://accumulo-hdfs-namenode-0.accumulo-hdfs-namenodes:8020/accumulo/tables/+r/root_tablet

java.lang.NullPointerException

at 
org.apache.accumulo.server.tablets.TabletTime.getInstance(TabletTime.java:60)

at org.apache.accumulo.tserver.tablet.Tablet.(Tablet.java:341)

at 
org.apache.accumulo.tserver.TabletServer$AssignmentHandler.run(TabletServer.java:2504)

at 
org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)

at java.lang.Thread.run(Thread.java:750)

2023-02-21 18:04:09,374 WARN [Root tablet assignment retry] 
[org.apache.accumulo.tserver.TabletServer]: failed to open tablet +r<< 
reporting failure to master

2023-02-21 18:04:09,374 WARN [Root tablet assignment retry] 
[org.apache.accumulo.tserver.TabletServer]: rescheduling tablet load in 600.00 
seconds

2023-02-21 18:07:03,210 WARN [SimpleTimer-0] 
[org.apache.accumulo.server.ServerUtil]: System swappiness setting is greater 
than ten (30) which can cause time-sensitive operations to be delayed. Accumulo 
is time sensitive because it needs to maintain distributed lock agreement.





The installed configuration is made up of 2 instances of HDFS data nodes, 2 
instances of T-SERVERs, 1 instance each of HDFS name node and Accumulo master.



> config -t accumulo.root

.

default| table.file.replication  | 0

site   |@override .. | 2

table  |@override .. | 5

.

.

> config -t accumulo.metadata

.

default| table.file.replication  | 0

site   |@override .. | 2

table  |@override .. | 5

.



How do I fix this problem?

Thanks in advance.

Ranga

RE: Problem creating tables in Accumulo

2023-02-22 Thread dlmarion 
It looks like the issue is that `maxTime` is not being set at [0], so `time`
is not being set at [1]. If I had to guess, Accumulo is having an issue
talking with ZooKeeper. Were there errors in the log when initializing
Accumulo? Are you able to `exec` into one of the Accumulo Kubernetes pods
and confirm connectivity with Accumulo?

[0]
https://github.com/apache/accumulo/blob/1.10/server/tserver/src/main/java/or
g/apache/accumulo/tserver/tablet/TabletData.java#L159
[1]
https://github.com/apache/accumulo/blob/1.10/server/tserver/src/main/java/or
g/apache/accumulo/tserver/tablet/TabletData.java#L166

 

From: Samudrala, Ranganath [USA] via user  
Sent: Tuesday, February 21, 2023 1:43 PM
To: Samudrala, Ranganath [USA] via user 
Subject: Problem creating tables in Accumulo

 

Hello,

 

Accumulo is running in Kubernetes cluster. Metdata data is stored in HDFS
and table data is stored in Amazon S3. No attempt has been made yet to
store/insert table data. But, when I attempt to create a table in Accumulo
via Java API, I see the following error.

 

 

WARN : org.apache.accumulo.core.clientImpl.ThriftTransportPool - Thread
"main" stuck on IO to accumulo-masters-0.accumulo-masters.acc
umulo-k8s.svc.cluster.local: (0) for at least 120033 ms

 

I see the below error in T-SERVER logs:

2023-02-21 18:04:09,373 WARN [Root tablet assignment retry]
[org.apache.accumulo.tserver.TabletServer]: exception trying to assign
tablet +r<<
hdfs://accumulo-hdfs-namenode-0.accumulo-hdfs-namenodes:8020/accumulo/tables
/+r/root_tablet
java.lang.NullPointerException
at
org.apache.accumulo.server.tablets.TabletTime.getInstance(TabletTime.java:60
)
at org.apache.accumulo.tserver.tablet.Tablet.(Tablet.java:341)
at
org.apache.accumulo.tserver.TabletServer$AssignmentHandler.run(TabletServer.
java:2504)
at
org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
at java.lang.Thread.run(Thread.java:750)
2023-02-21 18:04:09,374 WARN [Root tablet assignment retry]
[org.apache.accumulo.tserver.TabletServer]: failed to open tablet +r<<
reporting failure to master
2023-02-21 18:04:09,374 WARN [Root tablet assignment retry]
[org.apache.accumulo.tserver.TabletServer]: rescheduling tablet load in
600.00 seconds
2023-02-21 18:07:03,210 WARN [SimpleTimer-0]
[org.apache.accumulo.server.ServerUtil]: System swappiness setting is
greater than ten (30) which can cause time-sensitive operations to be
delayed. Accumulo is time sensitive because it needs to maintain distributed
lock agreement.

 

 

The installed configuration is made up of 2 instances of HDFS data nodes, 2
instances of T-SERVERs, 1 instance each of HDFS name node and Accumulo
master.

 

> config -t accumulo.root

.

default| table.file.replication  | 0 

site   |@override .. | 2

table  |@override .. | 5

.

.

> config -t accumulo.metadata

.

default| table.file.replication  | 0 

site   |@override .. | 2

table  |@override .. | 5

.

 

How do I fix this problem?


Thanks in advance.

Ranga