24hr CI with agitation also just finished successfully for me:
org.apache.accumulo.test.continuous.ContinuousVerify$Counts
REFERENCED=6598144114
UNREFERENCED=2000263
Michael Wall wrote:
Did my tweaks fix the count? Short answer is not yet.
After giving mappers and reducers 2048m, setting mapreduce.task.io.sort.mb
to 1024 and reducing the number of mappers and reducers from 264 to 140, I
reran with agitation yesterday. Josh had a good run without agitation and
I had removed the ci table already. On the run-verify, enough reducers ran
out of memory that the job failed. I have not gotten this work yet, was
hoping to talk to Keith when gets back.
On Fri, Aug 19, 2016 at 3:13 PM, Mike Drob<md...@mdrob.com> wrote:
Did this fix your count discrepancy?
On Thu, Aug 18, 2016, 10:53 AM Michael Wall<mjw...@gmail.com> wrote:
Ok, the run-verify MR took over 5 hours. I think muchos needs some
tweaks
in yarn settings.
On Thu, Aug 18, 2016 at 11:50 AM, Josh Elser<josh.el...@gmail.com>
wrote:
Uhh, woah. That's a rather big discrepancy. Yes, the number of
referenced+unreferenced should be equal to the number of entries in the
table (exactly, IIRC).
Michael Wall wrote:
The accumulo monitor showed 22.86 B entries in the ci table. Should
that
match the verify output?
On Thu, Aug 18, 2016 at 11:38 AM, Josh Elser<josh.el...@gmail.com>
wrote:
I didn't time it closely, but ingest ran for about 24hrs.
3B entries seems low for 8nodes, but I'm sure it's fine. I also had
to
knock back the hdfs replication for the ci table from 3 to 2 after
about
12
hrs because I was running out of space. That probably also helped my
run
go
faster :)
Michael Wall wrote:
Cool, thanks Josh. How long did you run? I ran 24 hours with 11
nodes
(8
tservers) on EC2 without agitation and got
REFERENCED=3118473398
UNREFERENCED=8002310
I started again with agitation last night and will check tonight.
On Thu, Aug 18, 2016 at 11:00 AM, Josh Elser<els...@apache.org>
wrote:
My verification on 5tservers with CI just finished successfully
without
agitation:
org.apache.accumulo.test.continuous.ContinuousVerify$
Counts
REFERENCED=7788662607
UNREFERENCED=2000310
I will be starting a run with agitation now.
Josh Elser wrote:
I retract my -1 (but not yet voting +1 though). It seems like when
fixed
ports are used and the tserver comes back online, the master will
resume
balancing. Not a blocker then.
Josh Elser wrote:
-1 due to https://issues.apache.org/jira/browse/ACCUMULO-4410.
This
appears to be a regression to me. It would stink for OPs as there
is
no
way to gracefully stop a TabletServer without also having to
restart
the
Master each time.
I still need to go through the rest of the diligence on rc2 (for
licensing stuff) as well as complete the verify.
Michael Wall wrote:
Accumulo Developers,
Please consider the following candidate for Accumulo 1.8.0.
Git Commit:
e5694cfa5ec3861e6f8f764e2b8477716e3072b9
Branch:
1.8.0-rc2
If this vote passes, a gpg-signed tag will be created using:
git tag -f -m 'Apache Accumulo 1.8.0' -s rel/1.8.0
e5694cfa5ec3861e6f8f764e2b8477716e3072b9
Staging repo:
https://repository.apache.org/content/repositories/orgapache
accumulo-1055
Source (official release artifact):
https://repository.apache.org/content/repositories/orgapache
accumulo-1055/org/apache/accumulo/accumulo/1.8.0/
accumulo-1.8.0-src.tar.gz
Binary:
https://repository.apache.org/content/repositories/orgapache
accumulo-1055/org/apache/accumulo/accumulo/1.8.0/
accumulo-1.8.0-bin.tar.gz
(Append ".sha1", ".md5", or ".asc" to download the
signature/hash
for
a
given artifact.)
All artifacts were built and staged with:
mvn release:prepare&& mvn release:perform
Signing keys are available at https://www.apache.org/dist/ac
cumulo/KEYS
(Expected fingerprint: 3E38C02F85303BAB37EE58C2F899DB
F0B8C3D406)
Release notes (in progress) can be found at:
https://accumulo.apache.org/release_notes/1.8.0
Please vote one of:
[ ] +1 - I have verified and accept...
[ ] +0 - I have reservations, but not strong enough to vote
against...
[ ] -1 - Because..., I do not accept...
... these artifacts as the 1.8.0 release of Apache Accumulo.
This vote will end on Fri Aug 19 16:30:00 UTC 2016
(Fri Aug 19 12:30:00 EDT 2016 / Fri Aug 19 09:30:00 PDT 2016)
Thanks!
P.S. Hint: download the whole staging repo with
wget -erobots=off -r -l inf -np -nH \
https://repository.apache.org/content/repositories/orgapache
accumulo-1055/
# note the trailing slash is needed
