Christopher, I don't remember configuring anything with regards to commons-vfs. How do I disable this?
After continuously hitting refresh on either monitor they just continuously say the manager is down. I"ll dig some more on Monday. Thanks for the response. -Vincent On Fri, Oct 13, 2023 at 5:39 PM Christopher <ctubb...@apache.org> wrote: > I am not sure what's up with the VFS error. I generally recommend > against using commons-vfs, because of the lack of any known good > versions, and the fact that most of what it is used for can be done > better in other ways. We have taken steps in recent versions to make > sure that the classloader stuff is more pluggable, so people do not > have to rely on commons-vfs, if they don't want to use it. One thing > that's concerning is that VFS is using some temp file in /tmp, which > could be wiped at any time by the OS, so if you do have need of VFS, > you may want to consider customizing it to use a different directory, > that isn't going to be wiped periodically by the OS. I believe the > relevant Accumulo property is: general.vfs.cache.dir > > The manager is down message is expected on initial startup if the > monitor starts faster than the manager, but refreshing the page should > make it work eventually if the manager is actually running. If that > doesn't happen eventually, I'm not sure what could be happening > without further troubleshooting. > > On Thu, Oct 12, 2023 at 12:36 PM Vincent Russell > <vincent.russ...@gmail.com> wrote: > > > > Christopher, > > > > I have run through an upgrade with a bit more data than I did previously > > and monitors with high availability just don't seem as consistent as they > > were with 2.0.1. > > > > After upgrading both accumulo monitors are reporting "Manager is Down" > and > > I was only able to achieve that after restarting both of the monitors and > > managers multiple times. > > > > The only ERROR log that I see in the manager logs are (at startup): > > > > java.nio.File.NoSuchFileException: /tmp/accumulo-vfs-cache-410@hostname > > .... > > at > > > org.accumulo.start.classloader.vfs.AccumuloVFSClassLoader.close(AccumuloVFSClassloader.java:453) > > > > I see this on both managers. > > > > Any ideas? > > > > Thanks, > > > > On Mon, Sep 25, 2023 at 12:21 PM Christopher <ctubb...@apache.org> > wrote: > > > > > I'm not sure about the monitor behavior. I've never had a reason to > > > run more than one, and I don't think we have any well-defined behavior > > > for trying to run more than one (I could see an argument for both). It > > > would be weird if one incorrectly reported "Master[now Manager] is > > > down" because another monitor was running, though. I would expect the > > > 2.1 behavior you described over that 2.0 behavior. > > > > > > On Mon, Sep 25, 2023 at 11:59 AM Vincent Russell > > > <vincent.russ...@gmail.com> wrote: > > > > > > > > Thank you Christopher, > > > > > > > > I did neglect to update the log4j2 files. > > > > > > > > One thing that is weird is that the monitors appear to work > differently > > > > with high availability. > > > > > > > > With 2.0.1 the accumulo monitors both work and you can hit them both > (one > > > > just says 'Master is down'; however, with 2.1.1, it appears that > only one > > > > works. Is this expected? > > > > > > > > Thank you, > > > > > > > > On Thu, Sep 14, 2023 at 4:28 PM Christopher <ctubb...@apache.org> > wrote: > > > > > > > > > I do not know about the error message you saw. > StatusConsoleListener > > > > > is not an Accumulo class, but it looks like it's a log4j one. My > best > > > > > guess is that during your upgrade, you did not migrate your log4j > > > > > config files over to be based on the log4j2 ones we have as an > example > > > > > in the 2.1 tarball. It's possible that there's some log4j > > > > > configuration issue that's preventing you from seeing the reason > why > > > > > the Monitor failed to start. > > > > > > > > > > To troubleshoot, I would compare your log4j config with what we > ship > > > > > in the 2.1 tarball and also check to see if you've done any log4j > > > > > config overrides in the system properties or environment variables > > > > > that control log4j. > > > > > > > > > > On Tue, Sep 12, 2023 at 8:10 PM Vincent Russell > > > > > <vincent.russ...@gmail.com> wrote: > > > > > > > > > > > > Thank you Christopher, > > > > > > > > > > > > Don't worry. I definitely planned on testing an upgrade on a > test > > > > > > cluster. :) > > > > > > > > > > > > I just upgraded a test cluster and everything appeared to go > smoothly > > > > > > except for one thing. > > > > > > > > > > > > I am able to login to the accumulo shell and I can see the table > > > that I > > > > > > created before the upgrade exists and I can scan that table. > > > > > > > > > > > > Before the upgrade the monitor was running on port 9995, but now > > > nothing > > > > > > appears to be running on that port. Nothing in the monitor log > > > > > exception: > > > > > > > > > > > > ERROR StatusConsoleListener Unable to send HTTP in appender > > > [MonitorLog] > > > > > > IllegalARgumentException: invalid URI schema <<hostname>> > > > > > > > > > > > > > > > > > > This seems not great, but I am not sure if it's related. > > > > > > > > > > > > It doesn't look like the monitor is running on another port. > Any > > > > > > suggestions for debugging this? > > > > > > > > > > > > Thanks, > > > > > > Vincent > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Aug 30, 2023 at 7:09 PM Christopher <ctubb...@apache.org > > > > > wrote: > > > > > > > > > > > > > My understanding is that the upgrade of ZK is pretty easy... > but I > > > > > > > would consult the ZooKeeper community for advice on that, since > > > they > > > > > > > are the experts. For Accumulo's part, I believe 2.0 worked fine > > > with > > > > > > > newer versions of ZK (3.5+), so you should be able to update ZK > > > first > > > > > > > if you didn't want to do them at the same time. There may be > some > > > > > > > things you might need to do to ensure your classpath is set up > > > > > > > correctly for Accumulo if using it with 2.0, since 2.0 might > > > assume ZK > > > > > > > 3.4 jar locations. I don't recall the specifics, but I > remember it > > > > > > > wasn't that bad. fluo-uno had used newer ZK versions with 2.0 > for a > > > > > > > while. The necessary config changes to support that might even > > > still > > > > > > > be scripted in its repo. > > > > > > > > > > > > > > I'm really not familiar with Hadoop upgrades, but as far as > > > Accumulo > > > > > > > is concerned, it would probably treat both 3.3.1 and 3.3.5 as > > > > > > > essentially equivalent. I don't think anything substantially > > > changed > > > > > > > that would cause Accumulo to even notice a difference. Seems > like > > > it > > > > > > > would just be a quick shutdown, update the classpath, restart, > for > > > > > > > Accumulo. For Hadoop, you may have some upgrade tasks... I'd > check > > > > > > > with the Hadoop community first. You could probably upgrade > Hadoop > > > > > > > before or after you upgrade Accumulo for this one. I don't > think > > > it's > > > > > > > going to matter much. > > > > > > > > > > > > > > If you're concerned about problems you might encounter, I > would run > > > > > > > through the upgrades on a small test instance first, just to > > > practice. > > > > > > > If you run into a specific problem doing that, we could > probably > > > offer > > > > > > > more specific insight, having probably seen it before. > > > > > > > > > > > > > > > > > > > > > On Wed, Aug 30, 2023 at 4:50 PM Vincent Russell > > > > > > > <vincent.russ...@gmail.com> wrote: > > > > > > > > > > > > > > > > Thank you Christopher. > > > > > > > > > > > > > > > > Does there exist some documentation for upgrading zookeeper > from > > > > > 3.4.14 > > > > > > > to > > > > > > > > 3.8.1? Is there a preferred upgrade path? > > > > > > > > > > > > > > > > Also...Is there any documentation on how this affects hadoop > when > > > > > using > > > > > > > > high availability? We are using hadoop 3.3.1. I want to > > > upgrade to > > > > > > > > hadoop 3.3.5, but I didn't know if that should be done as > part of > > > > > > > upgrading > > > > > > > > accumulo or later. > > > > > > > > > > > > > > > > Any experience folks have would be greatly appreciated. > > > > > > > > > > > > > > > > Thank you, > > > > > > > > Vincent > > > > > > > > > > > > > > > > On Thu, Aug 24, 2023 at 12:58 PM Christopher < > > > ctubb...@apache.org> > > > > > > > wrote: > > > > > > > > > > > > > > > > > The correct link seems to be > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://accumulo.apache.org/docs/2.x/troubleshooting/zookeeper#zookeeper-acls > > > > > > > > > > > > > > > > > > It looks like both `bin/accumulo dump-zoo` and > > > `bin/accumulo-util > > > > > > > > > dump-zoo` will do the same thing. It was originally a > > > disconnected > > > > > > > > > utility that existed for convenience, but was incorporated > > > into our > > > > > > > > > regular tooling recently as part of an effort to make all > these > > > > > > > > > utilities more easily discoverable and executable. I don't > > > think we > > > > > > > > > realized that it already had a convenient and documented > entry > > > > > point. > > > > > > > > > We were just looking for utilities with a "main" method. > Either > > > > > should > > > > > > > > > work fine. They call the same code. > > > > > > > > > > > > > > > > > > On Thu, Aug 24, 2023 at 11:04 AM Vincent Russell > > > > > > > > > <vincent.russ...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > Also the link to: > > > > > > > > > > > > > > > > > > > > > > > > > > https://accumulo.apache.org/docs/2.x/troubleshooting/ZooKeeper#ACLs > > > > > > > > > > > > > > > > > > > > is not resolving. > > > > > > > > > > > > > > > > > > > > What needs to be done at this step? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Vincent > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Aug 24, 2023 at 10:36 AM Vincent Russell < > > > > > > > > > vincent.russ...@gmail.com> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > > > > > > > I'm practicing going to the upgrade instructions from > > > 2.0.1 to > > > > > > > 2.1.1 > > > > > > > > > and I > > > > > > > > > > > wanted to confirm that the zoo-dump command is run via > the > > > > > > > > > accumulo-util > > > > > > > > > > > command and not via accumulo. > > > > > > > > > > > > > > > > > > > > > > The instructions say: > > > > > > > > > > > > > > > > > > > > > > $ACCUMULO_HOME/bin/accumulo dump-zoo --xml --root > > > /accumulo | > > > > > tee > > > > > > > > > > > PATH_TO_SNAPSHOT > > > > > > > > > > > > > > > > > > > > > > Thank you, > > > > > > > > > > > Vincent > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
