You are hitting the zookeeper timeout, default 30s I believe. You said you
are not oversubscribed for memory, but what about CPU? Are you running YARN
processes on the same nodes as the tablet servers? Is the tablet server
being pushed into swap or starved of CPU?
-Original Message-
From:
What does your classpath settings look like in accumulo-site.xml. I recently
made some fixes in 1.6.1-Snapshot where the context classloader was not being
used in all cases. I dont think this case was affected though.
Sent via the Samsung GALAXY S®4, an AT&T 4G LTE smartphone
Original
HOME/[^.].*.jar,
$HADOOP_HOME/lib/[^.].*.jar,
Classpaths that accumulo checks for updates and class files.
When using the Security Manager, please remove the ".../target/classes/"
values.
On Sun, Jun 15, 2014 at 10:49 AM, dlmarion wrote:
What does
I believe the problem that you are running into is that because dfs.default.uri
was not specified, then fs.defaultFS was used to write entries to the
accumulo.root and accumulo.metadata tables. Suggest doing the following:
Update to the latest version of Accumulo 1.6.1-SNAPSHOT
set instance.df
Jeff,
Which version of Accumulo are you using? Most of the classpath settings are in
the accumulo-site.xml file.
- Original Message -
From: "Jeff Kubina"
To: user@accumulo.apache.org
Sent: Monday, August 25, 2014 3:17:17 PM
Subject: Re: (U) I do I tell Accumulo where the jars for
We fixed some issues recently with the classloader in 1.6.1, make sure you are
using an up-to-date snapshot version. The classloader was changed in version
1.5 and now supports hosting jars from various locations, including HDFS.
Additionally, it supports creating a table level classloader that
What Accumulo version?
What Hadoop version?
From: Calvin Feder [mailto:calvin.fe...@argyledata.com]
Sent: Friday, January 16, 2015 7:29 PM
To: user@accumulo.apache.org
Subject: Moving HDFS name node to another host in accumulo 1.6
We need to move the HDFS name node of our cluster to a new
In 1.6, accumulo.metadata entries are absolute URLs. You can change the
existing URLs using the instance.volume.replacments[1] parameter.
[1]
http://accumulo.apache.org/1.6/accumulo_user_manual.html#_instance_volumes_replacements
> -Original Message-
> From: webmas...@webmaster.ms [mai
Did you restart Accumulo after you copied the jar? The general.classpaths
locations are not monitored and loaded dynamically.
Original message From: Srikanth Viswanathan
Date:02/18/2015 8:42 PM (GMT-05:00)
To: user@accumulo.apache.org Cc: Subject:
Installing custom autho
via the getTopKey() and getTopValue() methods. [1] should be a simple example.
[1]
https://git-wip-us.apache.org/repos/asf?p=accumulo.git;a=blob;f=core/src/main/java/org/apache/accumulo/core/iterators/user/GrepIterator.java;h=043a729a778fc34d2ee87a0227056ffac81b7fe7;hb=refs/heads/master
-
https://github.com/ericnewton/accumulo-opentsdb
From: Ranjan Sen [mailto:ranjan_...@hotmail.com]
Sent: Monday, July 20, 2015 6:25 PM
To: user@accumulo.apache.org
Subject: TSDB on Accumulo row key
Hi All,
Is there something like TSDB (Time series database) on Accumulo?
Thanks
Ranj
Except for any server side iterator logic, table balancers, etc...
- Original Message -
From: "Jonathan Parise"
To: user@accumulo.apache.org
Sent: Wednesday, July 29, 2015 3:02:27 PM
Subject: RE: Question regarding java being the choice for accumulo
I’m not sure what the motivat
Hotspotting was the first thing that came to my mind with the proposed
balancer. The fservers don't keep all the K/V in memory. You are balancing
query and live ingest across your resources.
Original message
From: Eric Newton
Date: 07/29/2015 8:46 PM (GMT-05:00)
T
It sounds like you want to try and not minor compact during your ingest of your
data. Is that correct?
From: William Slacum [mailto:wsla...@gmail.com]
Sent: Thursday, July 30, 2015 8:10 PM
To: user@accumulo.apache.org
Subject: Re: How to control Minor Compaction by programming
See
http:/
It's not temporary files, it's any file that has been compacted away. If you
keep files around longer than { dfs.namenode.checkpoint.period}, then you have
a chance to recover in case your most recent checkpoint is corrupt.
- Original Message -
From: "James Hughes"
To: user@accumulo
you determine the
MTTR and then schedule your backups accordingly; a backup in case you are not
able to recover your database using the techniques in the current
documentation.
[1] https://github.com/dlmarion/raccovery
From: James Hughes [mailto:jn...@virginia.edu]
Sent: Monday, August
Check out the implementation of
https://issues.apache.org/jira/browse/ACCUMULO-3793. You could do something
similar and start your process with 'nice' instead of 'numactl'.
> -Original Message-
> From: Christopher [mailto:ctubb...@apache.org]
> Sent: Monday, August 17, 2015 7:15 PM
> To:
+1 to Netty. I have used it and like it, but I think you will have to write all
of the code yourself. Nifty (Facebook) could be an option to use Thrift and
Netty together. Finagle (Twitter) would be another alternative.
- Original Message -
From: "Christopher"
To: "Accumulo User Lis
Anything in the master log?
Original message
From: Scott
Date: 08/21/2015 6:15 PM (GMT-05:00)
To: user@accumulo.apache.org
Subject: Accumulo Monitor NullPointerException
Hello,
We have Accumulo 1.6.3 set up on the HortonWorks install with a basic cluster
of on
Dylan,
I am a little confused about whether you want to place this in the contrib
area or whether you want to create a sub-project as both are mentioned in your
proposal. Also, if you intend for this to be a sub-project, have you looked at
the incubator process? From what I understand gi
and purpose,
and there is large opportunity for synergy in that Graphulo development may
help Accumulo development and vice versa. We're in that happy middle spot where
a sub-project makes sense. That said, this is a community decision, and so I'm
open to other opinions.
Regards, Dy
Both will work, but I think the answer depends on the amount of data that you
will be querying over and your query latency requirements. I would include the
wikisearch[1] storage scheme into your list as well (k/v table + indices).
Then, personally, I would rate them in the following order as d
Hey Jeff,
Take a look at [1] and see if the zookeeper balance issue mentioned is
applicable.
Dave
[1] https://accumulo.apache.org/release_notes/1.6.2.html
Original message
From: Jeff Turner
Date: 09/10/2015 7:42 PM (GMT-05:00)
To: user@accumulo.apache.org
Subject
It’s likely that you created a table, inserted data with column visibilities,
but never gave the user the authorization tokens to see the data. Try setting
the authorization tokens for the user with the setauths command. Since you are
using the ‘root’ user, it would be something like: setauths –
I don't have a script for you, but if you need to create one you could use the
script command in the shell to do something similar to the hbase script. Some
examples are in the comments in jira[1]. If you can figure out how you want the
table split, and it can be scripted, I might have time thi
BLUF: There exists the possibility of data loss when performing DataNode
decommissioning with Accumulo running. This note applies to installations of
Accumulo 1.5.0+ and Hadoop 2.5.0+.
DETAILS: During DataNode decommissioning it is possible for the NameNode to
report stale block locations (HD
Known issue in the release notes on the web page? We would have to update every
version though. Seems like we need a known issues document that lists issues in
dependencies that transcend Accumulo versions.
- Original Message -
From: "Josh Elser"
To: d...@accumulo.apache.org
Cc: use
I believe the RFile is assigned to each tablet that contain keys in the RFiles
key range. So, 1 RFile is assigned to one or more tablets. The bulk import
RFile is removed after it has been compacted by all assigned tablets.
- Original Message -
From: "Jeff Kubina"
To: user@accumulo.a
Are you trying to shut the whole system down, or just a couple of tablet
servers?Is your application reading and writing from/to Accumulo during this
time?
Original message
From: Denis
Date: 10/22/2015 6:03 PM (GMT-05:00)
To: user@accumulo.apache.org
Subject: Re:
ed.
> Yes, it is a production system with a lot of reads and writes.
>
> On 10/22/15, dlmarion wrote:
> >
> >
> > Are you trying to shut the whole system down, or just a couple of
> > tablet servers?Is your application reading and writing from/to
> > Accu
Can you provide the relevant classpath sections of your accumulo-site.xml file?
> -Original Message-
> From: Rob Povey [mailto:r...@maana.io]
> Sent: Thursday, October 29, 2015 8:01 PM
> To: user@accumulo.apache.org
> Subject: Accumulo Iterator painful development because TS don't pick up
So, without seeing your configuration, I would suggest trying something before
upgrading to 1.7. In 1.5 we changed the classloader so that it could load from
different locations. At the same time, we added the concept of classloader
contexts which are basically names for locations for jars. Tabl
Try replacing the vfs jar in lib with a 2.1-SNAPSHOT. Several issues have been
fixed, but one of them is that if more than one monitored resource changed then
it would miss some of them.
- Original Message -
From: "Rob Povey"
To: user@accumulo.apache.org
Sent: Friday, October 30, 2
Also, turn the logging on the tservers up to DEBUG for
org.apache.accumulo.start.classloader.*. You should see a line in the log that
starts with "monitoring "
- Original Message -
From: dlmar...@comcast.net
To: user@accumulo.apache.org
Sent: Friday, October 30, 2015 12:22:53 PM
Su
Suggest replacing the VFS 2.0 jar in the Accumulo lib directory with a VFS
2.1-SNAPSHOT jar from [1]. There are some bugs that have been fixed, but VFS
2.1 has not been released yet. I did this and have not had many issues loading
from HDFS since. Loading jars from file:/// should work, I believ
try: ./accumulo -add shell
- Original Message -
From: "Christopher"
To: user@accumulo.apache.org, "Sravankumar Reddy Javaji"
Sent: Monday, March 7, 2016 4:28:03 PM
Subject: Re: Class path for shell commands
Are you using a 1.6.x or earlier version? I think we fixed a bug in 1.7
Another option is to drop your jar into $ACCUMULO_HOME/lib/ext on the node
where you will be running the shell.
- Original Message -
From: "Sravankumar Reddy Javaji (BLOOMBERG/ 731 LEX)"
To: user@accumulo.apache.org
Sent: Monday, March 7, 2016 4:31:49 PM
Subject: Re: Class path for
Some other thoughts in addition to the sharding:
1. Are your tablets spread out evenly across your tablet servers?
2. How many threads are you using in your batch scanner?
3. What is the table.scan.max.memory setting?
From: Andrew Hulbert [mailto:ahulb...@ccri.com]
Sent: Sunday, April 1
Timely is a time series database application that provides secure access to
time series data. It is designed to be used with Apache Accumulo for
persistence and Grafana for visualization. Timely is located at
https://github.com/NationalSecurityAgency/timely .
+1 good catch Josh
- Original Message -
From: "Josh Elser"
To: user@accumulo.apache.org
Sent: Thursday, July 7, 2016 2:12:42 PM
Subject: Re: java.lang.NoClassDefFoundError with fields of custom Filter
Beware using the HDFS classloader in any Accumulo release that does not
contain
The batch writer has several knobs (latency time, memory buffer, etc) that you
can tune to meet your requirements. The values for those settings will depend
on a lot of variables, to include:
- number of tablet servers
- size of mutations
- desired latency
- memory buffer
- configuration se
Doesn't this use the 6 batch scanners serially?
- Original Message -
From: "Sven Hodapp"
To: "user"
Sent: Wednesday, August 24, 2016 11:56:14 AM
Subject: Re: Accumulo Seek performance
Hi Josh,
thanks for your reply!
I've tested your suggestion with a implementation like that:
Calling BatchScanner.iterator() is what starts the work on the server side. You
should do this first for all 6 batch scanners, then iterate over all of them in
parallel.
- Original Message -
From: "Sven Hodapp"
To: "user"
Sent: Thursday, August 25, 2016 4:53:41 AM
Subject: Re: Ac
But does toList exhaust the first iterator() before going to the next?
- Dave
- Original Message -
From: "Sven Hodapp"
To: "user"
Sent: Thursday, August 25, 2016 9:42:00 AM
Subject: Re: Accumulo Seek performance
Hi dlmarion,
toList should also call ite
The internal metrics are exposed via JMX. You could connect to each server and
pull the metrics. Or, you could use the Hadoop Metrics2 framework and push the
metrics to some listening service. If you choose the latter route, then you can
configure Hadoop also and get its metrics. An example of c
What was happening on that 1 tserver? Was it in garbage collection? Was it
having network or O/S issues?
- Original Message -
From: "Michael Moss (BLOOMBERG/ 731 LEX)"
To: user@accumulo.apache.org
Sent: Friday, September 9, 2016 9:40:42 AM
Subject: 1 of 20 TServers unresponsive/slow
We have seen this before: a tserver that is hosting metadata tablets has issues
and starts causing problems within the cluster. You could try using the
HostRegexTableLoadBalancer[1,2] to segregate your metadata tablets from the
other tables. This doesn't fully eliminate the SPOF, but it should
You can also use the admin dumpConfig command to dump all of the configuration
information, then look at the user permission files.
From: Christopher [mailto:ctubb...@apache.org]
Sent: Friday, September 23, 2016 5:39 PM
To: user@accumulo.apache.org
Subject: Re: how do I list user permissions
Congrats!
- Original Message -
From: "Mike Walch"
To: user@accumulo.apache.org
Sent: Friday, October 14, 2016 3:21:00 PM
Subject: [ANNOUNCE] Apache Fluo 1.0.0-incubating released
The Apache Fluo (incubating) team is happy to announce the release of Fluo
1.0.0-incubating:
https:/
I don't remember seeing anything about native library performance improvement,
but I do know that there has been talk of a performance regression in Accumulo
1.7.0 using the default configuration. I don't remember the specific details
off-hand, but it had something to do with the default value o
So if I understand this correctly, for this use case, you could do the
following:
courseId studentId
For either of your queries (1 and 2 below) you could use a BatchScanner with
the set of Ranges being the course ids from input C. In your client you would
add the resulting columnFamily (stu
For those interested, we released the third version of Timely today which
includes several feature changes and performance improvements. Release notes
are at: https://nationalsecurityagency.github.io/timely/docs/#release-notes
>>> Is Accumulo able to import these files, considering that they are two
>>> different locality groups
Yes.
>>> without triggering a huge major compaction?
Depends on your table.compaction.major.ratio and table.file.max settings.
Sorry, not a real answer, but I think the answer is "it de
Regarding #2 – I think there are two options here:
1. Modify Accumulo to take advantage of HDFS Heterogeneous Storage
2. Modify Accumulo WAL code to support volumes
From: Jeff Kubina [mailto:jeff.kub...@gmail.com]
Sent: Wednesday, November 02, 2016 9:02 PM
To: user@accumulo.apache.org
Sub
It used[1] to be in the documentation when it was hosted on the monitor. I did
not see it looking at the current documentation. Looks like it was lost (and
[1] is likely now out of date).
[1] https://github.com/apache/accumulo/blob/1.4.0/docs/metrics.html
- Original Message -
From:
What kind of changes are you looking to alert off? Is it data changes or
configuration changes?
> -Original Message-
> From: vaibhav thapliyal [mailto:vaibhav.thapliyal...@gmail.com]
> Sent: Tuesday, November 22, 2016 7:20 AM
> To: user@accumulo.apache.org
> Subject: Detecting database cha
In one case, the tserver will send data back to the client when it fills its
buffer. When this happens, it's possible that the iterator could be torn
down and re-seeked to the last key returned. You could increase the size of
this buffer to see if that helps
(http://accumulo.apache.org/1.8/accumulo
Regarding the referenced paper, pre-splitting the tables, using an optimized
zookeeper deployment, and increasing concurrent minor / major compactions are
good things. I'm not sure that we want to recommend turning off the write ahead
logs and replication for production deployments.
-Origin
Re #2: Does your Accumulo processes run as the hdfs user on the O/S, or as the
accumulo user? Make sure you are checking the correct users trash folder. Also,
check the Accumulo garbage collector log to see if the GC process deleted the
WAL files. Take a look at [1] to see if you are hitting thi
You can also use the tserver.walog.max.age property to ensure that the walogs
roll if there is no activity. The default is 24h and was backported to 1.7.2.
See ACCUMULO-4004 for more info.
-Original Message-
From: Josh Elser [mailto:els...@apache.org]
Sent: Friday, December 29, 2017 10:
This may suggest an issue with client, either getting the data to the client or
the client itself (although I think there are other performance related changes
you could make). I’m curious what the end goal is here. Is this a real world
use case? If you are using this type of benchmark to evalua
IIRC the 'service.name' property in the metrics properties file is used to
note from which service the metric was emitted. I would suggest testing this
by setting the service.name property to tserver1 and see if your metrics
show up with that name. If they do, then you might be able to modify the
s
The Monitor[1] gets a MasterMonitorInfo[2] object from the Master that contains
the information you need. The tableMap member contains a TableInfo[3] object
for each table that holds the information you are looking for.
[1]
https://github.com/apache/accumulo/blob/1.9/server/monitor/src/main/
Please see: https://lists.apache.org/thread/p7mwtkfpbyb551pw5k7yg61jopf50m8s
From: Josef Roehrl - PHEMI
Sent: Monday, June 20, 2022 7:02 PM
To: user@accumulo.apache.org
Subject: Re: Tablet Server Session Id Out of Range
Hi Jonathan,
We too have exactly this issue as of a couple of day
I believe that S3 Guard is OBE, but you still need to put the WAL on HDFS as S3
does not support sync. If you put your WAL in S3, and you have a tserver
failure, then it’s possible that you will lose data.
From: Christopher
Sent: Wednesday, October 12, 2022 4:12 PM
To: user@accumulo.apache.
Take a look at https://github.com/dlmarion/accumulo-k8s. You can put the
Accumulo config in a ConfigMap (accumulo-config.yaml), then mount it at the
correct location in another container (see the server process yaml files). I
used a job to do the init phase, see accumulo-init.yaml.
From
It looks like the issue is that `maxTime` is not being set at [0], so `time`
is not being set at [1]. If I had to guess, Accumulo is having an issue
talking with ZooKeeper. Were there errors in the log when initializing
Accumulo? Are you able to `exec` into one of the Accumulo Kubernetes pods
and c
Does this apply? https://bugs.openjdk.org/browse/JDK-8190917
From: Sanjay Deshmukh
Sent: Monday, June 5, 2023 3:28 PM
To: user@accumulo.apache.org
Subject: Re: Impact from Enabling TLS
Sure thing. We're running Accumulo 1.10.1. Our configuration has 15 tablet
servers and a client pool of
Bruce,
There are no future planned releases for the 1.x line. In 2.1 Scan
Executors[1] offer a way for the user to configure the resource pools used
for scanning a table. You could use a prioritizer to influence the execution
order of scans in the default scan executor, or setup different sca
My guess is that the error is happening on the server side and being raised
at
https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/util/Base64.java#L713.
If there is a stack trace in the server log, then that would help pinpoint the
issue. I’m guessing here, but it
Matt,
Did you see Eric Newton's response yesterday? Running on a ram disk has been
done; however minor and major compactions will still occur.
- Dave
- Original Message -
From: "Matthew J. Moore"
To: user@accumulo.apache.org
Sent: Wednesday, September 12, 2012 12:32:31
The number of entries will show up on the monitor after a compaction.
Dave
- Original Message -
From: "ameet kini"
To: user@accumulo.apache.org
Sent: Friday, September 21, 2012 10:42:32 AM
Subject: bulk ingested table showing zero entries on the monitor page
I'm ingest
Krishmin,
In the wikisearch example there is a non-sharded index table and a sharded
document table. The index table is used to reduce the number of tablets that
need to be searched for a given set of terms. Is your setup similar? I'm a
little confused since you mention using a sharded index t
John,
Where is this VM located?
Dave
- Original Message -
From: "John Vines"
To: user@accumulo.apache.org
Sent: Friday, November 30, 2012 3:17:48 PM
Subject: Accumulo VM
I recut an Accumulo-1.4.2 VM for people to have a quick start with Accumulo. It
is configured to
ACCUMULO-837
- Original Message -
From: "John Vines"
To: user@accumulo.apache.org
Sent: Thursday, February 7, 2013 12:23:48 PM
Subject: Re: upgrading to CDH3u5
I don't think that CDH3u5 requires a recompile, that's only the case for
CDH4/hadoop 0.23/hadoop2.
On Thu, Feb 7, 20
I think these might be more accurate descriptions based on the changes that
occurred to the code after the point at which that snippet was put into the
ACCUMULO-708 comments.
1. JVM s ystem c lasspath ---> JDK jars on the local server
2. JVM application classpath ---> Hadoop, ZK, acc
I believe that Brian Loss committed code in 1.5 for a column visibility
correction iterator or something that you could use to do this. You could use
that and compact the table after the import.
- Original Message -
From: "Donald Miner"
To: user@accumulo.apache.org
Sent: Tuesday, Jul
You can create a custom load balancer for your table. I think you said that
your tables are organized by date. If the hot spot are contiguous days, then
you could write a load balancer that does a round robin assignment.
Sent via the Samsung GALAXY S®4, an AT&T 4G LTE smartphone
Origi
You can set it in the shell on the table. Just override the default tablet
balancer for the table. I think the master has to use the Table load balancer
also if it is not set by default.
- Original Message -
From: "David M. Slater"
To: user@accumulo.apache.org
Sent: Wednesday, Augus
Data is available immediately. The counts on the monitor wont update until a
compaction occurs.
Sent via the Samsung GALAXY S®4, an AT&T 4G LTE smartphone
Original message
From: "Dickson, Matt MR"
Date: 10/17/2013 8:35 PM (GMT-05:00)
To: "'user@accumulo.apache.org'"
Sub
+1
From: Keith Turner [mailto:ke...@deenlo.com]
Sent: Thursday, November 14, 2013 3:42 PM
To: d...@accumulo.apache.org; user@accumulo.apache.org
Subject: [VOTE] Deprecate mock in 1.6.0
Should we deprecate mock accumulo for 1.6.0? This was considered [1] for
1.5.0. I started thinking abou
There is a dumpconfig command in 1.6.0. The source should help you:
https://github.com/apache/accumulo/blob/1.6.0-SNAPSHOT/server/base/src/main/
java/org/apache/accumulo/server/util/Admin.java. Look at the printConfig
method.
From: Sujee Maniyam [mailto:su...@sujee.net]
Sent: Tuesday, December
Have you tried killing one or both of the tservers?
Sent via the Samsung GALAXY S®4, an AT&T 4G LTE smartphone
Original message
From: Anthony F
Date:01/25/2014 10:12 AM (GMT-05:00)
To: user@accumulo.apache.org
Subject: Found two locations for the same extent
I have a ta
s on
!METADATA to root and was able to delete one of the offending entries. At that
point, the system stabilized. Any tips on avoiding getting into this situation?
On Sat, Jan 25, 2014 at 10:51 AM, dlmarion wrote:
Have you tried killing one or both of the tservers?
Sent via the Samsung GALAXY S®4
You will also want to set the replication on the directory/jars in HDFS to
something high. I think in my testing I could not set it above 50.
-Original Message-
From: Josh Elser [mailto:josh.el...@gmail.com]
Sent: Thursday, February 20, 2014 12:30 PM
To: user@accumulo.apache.org
Subject:
The master runs a load balancer.
Sent via the Samsung GALAXY S®4, an AT&T 4G LTE smartphone
Original message
From: "Kepner, Jeremy - 0553 - MITLL"
Date:03/02/2014 8:34 PM (GMT-05:00)
To: user@accumulo.apache.org
Subject: pre-split table tablet server assignment
If I pre
We had an issue in our testing
(https://issues.apache.org/jira/browse/ACCUMULO-2480). The root cause was a
misconfiguration for automatic failover. The sshfence feature does not
handle network failures, so you have to configure it with the
shell(/bin/true) command also (separated by a newline, unli
How many threads are you using in the AccumuloOutputFormat? What is your
latency set to?
From: Adam Fuchs [mailto:afu...@apache.org]
Sent: Tuesday, April 08, 2014 5:36 PM
To: user@accumulo.apache.org
Subject: Re: Advice on increasing ingest rate
MIke,
What version of Accumulo are yo
88 matches
Mail list logo
