I reran the bulk import+agitation test that found #1231 after applying fixes for it. During that test I found #1260 [1], which I am looking into fixing. For the latest test I imported 1,000 dirs each with 10M entries and 48 files. One bulk imports failed because of #1260. I ran verification and all other bulk imports were ok. The only missing data was from the single bulk import that failed. I confirmed this by determining that all missing data had a UUID related to the failed bulk import.
Hopefully I can figure out #1260 soon. I would like to fix it before RC2 if possible. [1] : https://github.com/apache/accumulo/issues/1260 On Tue, Jun 25, 2019 at 3:58 PM Mike Miller <[email protected]> wrote: > > Successfully ran "mvn clean verify -Psunny" on the 2.0 Branch. > > I have been running Random walk tests against 2.0 branch and found > something strange with one of the tests. I opened an issue in the testing > repo: > https://github.com/apache/accumulo-testing/issues/83 > > Hopefully it is just something with the test itself but it has me worried. > The one Bulk Import test tries to create directories in HDFS before calling > bulk import on them. The bulk import then fails because the directory > doesn't exist yet. This happens after I confirm with FS that the file > exists in HDFS. I will keep digging but it may have something to do with > the changes from the new Bulk import method and extra validation that was > added. > > On Mon, Jun 24, 2019 at 6:52 PM Christopher <[email protected]> wrote: > > > This vote fails with: 4 binding -1s. > > > > Thanks everyone who voted and tested (whether your vote was binding or > > not). I'll do another release candidate after #1231 is fixed, but > > please feel free to continue testing and identifying more issues that > > should be fixed for the next release candidate. > > > > (Note: There's a U.S. holiday next week, which may impact some > > people's availability and the timing of release voting. The minimum of > > 72 hours for any release votes still strictly applies[always], but if > > the vote isn't closed right away, it's probably because folks are > > spending time with friends and family.) > > > > > > On Mon, Jun 24, 2019 at 6:38 PM Christopher <[email protected]> wrote: > > > > > > -1 because of #1231 also. > > > > > > I will investigate > > https://github.com/apache/accumulo-maven-plugin/issues/4 > > > > > > On Mon, Jun 24, 2019 at 9:00 AM Owens, Mark <[email protected]> > > wrote: > > > > > > > > -1 > > > > Although my local unit and integration tests passed, I agree that > > #1231 should be fixed before releasing publicly. > > > > > > > > -----Original Message----- > > > > From: Mike Miller <[email protected]> > > > > Sent: Monday, June 24, 2019 8:47 AM > > > > To: [email protected] > > > > Subject: Re: [VOTE] Apache Accumulo 2.0.0-rc1 > > > > > > > > -1 > > > > So we can fix #1231 > > > > I was able to do some Random Walk testing using Uno on my laptop last > > week on the 2.0 branch. It was mostly successful. > > > > I found this RW bug I plan on fixing this week: > > > > https://github.com/apache/accumulo-testing/issues/83 > > > > I also plan on taking a look at #1231. > > > > > > > > On Sat, Jun 22, 2019 at 10:09 AM Jeffrey Manno < > > [email protected]> > > > > wrote: > > > > > > > > > Just giving an update on where my testing is at in case I forget. > > > > > > > > > > So far haven't noticed any errors. I have been running continuous > > > > > ingest on both a 10 tserver EC2 cluster and on my machine for roughly > > > > > ~17 hours. I do get the tracing spans being dropped warning on the > > Ec2 > > > > > side. Not sure of that cause of that as of yet. > > > > > > > > > > All the usual mvn build profiles have passed without error. > > > > > > > > > > > > > > > On Fri, Jun 21, 2019 at 4:19 PM Keith Turner <[email protected]> > > wrote: > > > > > > > > > > > -1 mainly because of #1231. Also would really like to be able to > > > > > > validate external projects can build against RC, but was unable to. > > > > > > > > > > > > I have been doing a lot testing this past week on EC2 using > > > > > > 2.0.0-SNAPSHOT built from 2ac7c7e. > > > > > > > > > > > > I successfully ran continuous ingest with agitation and multiple > > > > > > ingest patterns for 18 hrs on a 11 node cluster. I used the > > > > > > following instructions > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/accumulo-testing/blob/master/docs/ingest-tes > > > > > t.md > > > > > > > > > > > > Then I ran the new bulk version of continuous ingest. I generated > > a > > > > > > lot of files to bulk import, started agitation, and then started > > > > > > importing. I was using the following instructions > > > > > > > > > > > > > > > > > > > https://github.com/apache/accumulo-testing/blob/master/docs/bulk-test. > > > > > md > > > > > > > > > > > > While running this I found the following bug. > > > > > > > > > > > > https://github.com/apache/accumulo/issues/1231 > > > > > > > > > > > > I ran the new garbage collection simulation on a 6 node cluster > > with > > > > > > 15 generators and 1 collector for ~18hrs. I ran agitation for this > > > > > > test. I need to document how I ran this using docker swarm. > > > > > > > > > > > > > > https://github.com/apache/accumulo-testing/blob/master/docs/gcs.md > > > > > > > > > > > > I tried building Fluo against this RC, but was unable to because of > > > > > > the following bug. > > > > > > > > > > > > https://github.com/apache/accumulo-maven-plugin/issues/4 > > > > > > > > > > > > I found the following minor issues while testing. > > > > > > > > > > > > https://github.com/apache/accumulo-testing/issues/84 > > > > > > https://github.com/apache/accumulo/issues/1230 > > > > > > https://github.com/apache/accumulo/issues/1223 > > > > > > > > > > > > Thanks for putting this RC together Christopher. > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jun 20, 2019 at 11:42 PM Christopher <[email protected]> > > > > > wrote: > > > > > > > > > > > > > > Accumulo Developers, > > > > > > > > > > > > > > Well, this is exciting, and a long time coming! > > > > > > > Please consider the following candidate for Apache Accumulo > > 2.0.0. > > > > > > > > > > > > > > Git Commit: > > > > > > > a32ff67f1a455bc6cd21ef2854f8c1d1c0f7de98 > > > > > > > Branch: > > > > > > > 2.0.0-rc1 > > > > > > > > > > > > > > If this vote passes, a gpg-signed tag will be created using: > > > > > > > git tag -f -m 'Apache Accumulo 2.0.0' -s rel/2.0.0 \ > > > > > > > a32ff67f1a455bc6cd21ef2854f8c1d1c0f7de98 > > > > > > > > > > > > > > Staging repo: > > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapacheaccumulo-1 > > > > > 080 > > > > > > > Source (official release artifact): > > > > > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapacheaccumulo-1 > > > > > 080/org/apache/accumulo/accumulo/2.0.0/accumulo-2.0.0-src.tar.gz > > > > > > > Binary: > > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapacheaccumulo-1 > > > > > 080/org/apache/accumulo/accumulo/2.0.0/accumulo-2.0.0-bin.tar.gz > > > > > > > > > > > > > > Append ".asc" to download the cryptographic signature for a given > > > > > > artifact. > > > > > > > (You can also append ".sha1" or ".md5" instead in order to verify > > > > > > > the > > > > > > checksums > > > > > > > generated by Maven to verify the integrity of the Nexus > > repository > > > > > > > staging area.) > > > > > > > > > > > > > > Signing keys are available at > > > > > https://www.apache.org/dist/accumulo/KEYS > > > > > > > (Expected fingerprint: 8CC4F8A2B29C2B040F2B835D6F0CDAE700B6899D) > > > > > > > > > > > > > > In addition to the tarballs and their signatures, the following > > > > > checksum > > > > > > > files will be added to the dist/release SVN area after release: > > > > > > > accumulo-2.0.0-src.tar.gz.sha512 will contain: > > > > > > > SHA512 (accumulo-2.0.0-src.tar.gz) = > > > > > > > > > > > > > > > > > > > > 92b7a0910b2fd9c28f47af5ce66b52b515c5affb16ada0da7f20499fd552f647e0841e > > > > > 2c61e67b609016bb3702888e42646176742082cc9ead33f782a83d021d > > > > > > > accumulo-2.0.0-bin.tar.gz.sha512 will contain: > > > > > > > SHA512 (accumulo-2.0.0-bin.tar.gz) = > > > > > > > > > > > > > > > > > > > > f85e60c38d883045bd797e75288900408d85aee16ae53eabc76d0bb3f0daea6f51b9e3 > > > > > 5396c7a6c8ff5e19a4bcd668793f08772b42e6cf41e29b3d70476b95c1 > > > > > > > > > > > > > > Release notes (in progress) can be found at: > > > > > > > https://accumulo.apache.org/release/accumulo-2.0.0/ > > > > > > > > > > > > > > Release testing instructions: > > > > > > > https://accumulo.apache.org/contributor/verifying-release > > > > > > > > > > > > > > Please vote one of: > > > > > > > [ ] +1 - I have verified and accept... > > > > > > > [ ] +0 - I have reservations, but not strong enough to vote > > against... > > > > > > > [ ] -1 - Because..., I do not accept... > > > > > > > ... these artifacts as the 2.0.0 release of Apache Accumulo. > > > > > > > > > > > > > > This vote will remain open until at least Mon 24 Jun 2019 > > 04:00:00 > > > > > > > AM > > > > > > UTC. > > > > > > > (Mon 24 Jun 2019 12:00:00 AM EDT / Sun 23 Jun 2019 09:00:00 PM > > > > > > > PDT) Voting can continue after this deadline until the release > > > > > > > manager sends an email ending the vote. > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > P.S. Hint: download the whole staging repo with > > > > > > > wget -erobots=off -r -l inf -np -nH \ > > > > > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapacheaccumulo-1 > > > > > 080/ > > > > > > > # note the trailing slash is needed > > > > > > > > > > > > >
