Re: TLS Support in Accumulo

2023-10-17 Thread Logan Jones
Hey Christopher:

Thanks for the response. Yeah, some performance hit was expected, it was
just the magnitude that we weren't expecting.
Unfortunately, I don't have a great benchmark to tell you exactly what we
saw, though we may have those numbers in the coming weeks. We decided to
pursue wire encryption through another means, but I am hoping to review
some of the numbers at a later date.

- Logan

On Tue, Oct 17, 2023 at 12:42 AM Christopher  wrote:

> I think a performance hit is expected, due to the expected overhead of the
> TLS handshake, and the number of connections Accumulo requires in order to
> distribute work across a cluster. I think whether the overhead is tolerable
> is a per user decision, and may also be dependent upon the details of your
> application, table content, query patterns, hardware, and JVM support. I'm
> sure it's not suitable for everybody's use case, but could be a useful
> option in some circumstances. It's really hard to make general statements
> about whether it's worthwhile, though, because of different people having
> different requirements and environments.
>
> I am curious, though, if you could characterize the overhead you saw, as a
> point of comparison.
>
> On Mon, Oct 16, 2023, 23:18 Logan Jones  wrote:
>
> > Hello:
> >
> > I know that Accumulo has support for TLS. When turning on TLS support, we
> > noticed some pretty serious performance hits as a result of turning this
> on
> > in 1.10.2. Does anyone actually have TLS turned on for larger clusters?
> Are
> > there any known performance problems with turning on TLS?
> >
> > Thanks in advance,
> >
> > - Logan
> >
>


Re: accumulo upgrade from 2.0.1 to 2.1.1

2023-10-17 Thread Christopher
On Sat, Oct 14, 2023 at 9:41 AM Vincent Russell
 wrote:
>
> Christopher,
>
> I don't remember configuring anything with regards to commons-vfs.  How do
> I disable this?

There's some VFS stuff enabled by default, for backwards
compatibility. There isn't an explicit way to disable it. It's being
phased out, so that it's entirely optional, via a pluggable mechanism,
if somebody wants to use it. But I've never seen the default stuff
cause any errors like what you're seeing if you're not using it to
specify any non-local paths in your classpath properties.

>
> After continuously hitting refresh on either monitor they just continuously
> say the manager is down.
>
> I"ll dig some more on Monday.
>
> Thanks for the response.
>
> -Vincent
>
> On Fri, Oct 13, 2023 at 5:39 PM Christopher  wrote:
>
> > I am not sure what's up with the VFS error. I generally recommend
> > against using commons-vfs, because of the lack of any known good
> > versions, and the fact that most of what it is used for can be done
> > better in other ways. We have taken steps in recent versions to make
> > sure that the classloader stuff is more pluggable, so people do not
> > have to rely on commons-vfs, if they don't want to use it. One thing
> > that's concerning is that VFS is using some temp file in /tmp, which
> > could be wiped at any time by the OS, so if you do have need of VFS,
> > you may want to consider customizing it to use a different directory,
> > that isn't going to be wiped periodically by the OS. I believe the
> > relevant Accumulo property is: general.vfs.cache.dir
> >
> > The manager is down message is expected on initial startup if the
> > monitor starts faster than the manager, but refreshing the page should
> > make it work eventually if the manager is actually running. If that
> > doesn't happen eventually, I'm not sure what could be happening
> > without further troubleshooting.
> >
> > On Thu, Oct 12, 2023 at 12:36 PM Vincent Russell
> >  wrote:
> > >
> > > Christopher,
> > >
> > > I have run through an upgrade with a bit more data than I did previously
> > > and monitors with high availability just don't seem as consistent as they
> > > were with 2.0.1.
> > >
> > > After upgrading both accumulo monitors are reporting "Manager is Down"
> > and
> > > I was only able to achieve that after restarting both of the monitors and
> > > managers multiple times.
> > >
> > > The only ERROR log that I see in the manager logs are (at startup):
> > >
> > > java.nio.File.NoSuchFileException: /tmp/accumulo-vfs-cache-410@hostname
> > > 
> > > at
> > >
> > org.accumulo.start.classloader.vfs.AccumuloVFSClassLoader.close(AccumuloVFSClassloader.java:453)
> > >
> > > I see this on both managers.
> > >
> > > Any ideas?
> > >
> > > Thanks,
> > >
> > > On Mon, Sep 25, 2023 at 12:21 PM Christopher 
> > wrote:
> > >
> > > > I'm not sure about the monitor behavior. I've never had a reason to
> > > > run more than one, and I don't think we have any well-defined behavior
> > > > for trying to run more than one (I could see an argument for both). It
> > > > would be weird if one incorrectly reported "Master[now Manager] is
> > > > down" because another monitor was running, though. I would expect the
> > > > 2.1 behavior you described over that 2.0 behavior.
> > > >
> > > > On Mon, Sep 25, 2023 at 11:59 AM Vincent Russell
> > > >  wrote:
> > > > >
> > > > > Thank you Christopher,
> > > > >
> > > > > I did neglect to update the log4j2 files.
> > > > >
> > > > > One thing that is weird is that the monitors appear to work
> > differently
> > > > > with high availability.
> > > > >
> > > > > With 2.0.1 the accumulo monitors both work and you can hit them both
> > (one
> > > > > just says 'Master is down'; however, with 2.1.1, it appears that
> > only one
> > > > > works.Is this expected?
> > > > >
> > > > > Thank you,
> > > > >
> > > > > On Thu, Sep 14, 2023 at 4:28 PM Christopher 
> > wrote:
> > > > >
> > > > > > I do not know about the error message you saw.
> > StatusConsoleListener
> > > > > > is not an Accumulo class, but it looks like it's a log4j one. My
> > best
> > > > > > guess is that during your upgrade, you did not migrate your log4j
> > > > > > config files over to be based on the log4j2 ones we have as an
> > example
> > > > > > in the 2.1 tarball. It's possible that there's some log4j
> > > > > > configuration issue that's preventing you from seeing the reason
> > why
> > > > > > the Monitor failed to start.
> > > > > >
> > > > > > To troubleshoot, I would compare your log4j config with what we
> > ship
> > > > > > in the 2.1 tarball and also check to see if you've done any log4j
> > > > > > config overrides in the system properties or environment variables
> > > > > > that control log4j.
> > > > > >
> > > > > > On Tue, Sep 12, 2023 at 8:10 PM Vincent Russell
> > > > > >  wrote:
> > > > > > >
> > > > > > > Thank you Christopher,
> > > > > > >
> > > > > > > Don't worry.  I definitely planned on testing an upgrade on a
> >