Re: [Discuss] Introducing Flexible Authentication in Cassandra via Feature Flag

Mon, 12 Feb 2024 22:03:54 -0800

Auth is one of those things that needs to be a bit more concrete 

In the scenario you describe, you already have an option to deploy the auth in piece partially during the rollout (pause halfway through) in the cluster and look for asymmetric connections, and the option to drop in a new Authenticator jar in the class path that does the flexible auth you describe 

I fear that the extra flexibility this allows for 1% of operations exposes people to long term problems

Have you considered just implementing the feature flag you describe using the existing plugin infrastructure ?

On Feb 12, 2024, at 9:47 PM, Gaurav Agarwal <gaurav.iit...@gmail.com> wrote:


Dear Dinesh and Abe,

Thank you for reviewing the document on enabling Cassandra authentication. I apologize that I didn't initially include the following failure scenarios where this feature could be particularly beneficial (I've included them now):

Below are the failure scenarios:
  • Incorrect credentials: If a client accidentally uses the wrong username/password combination during the rollout, While restarting the server to enable authentication, it will refuse connections with incorrect credentials. This can temporarily interrupt the service until correct credentials are sent.
  • Missed service auth updates: In a large-scale system, a service "X" might miss the credential update during rollout. After some server nodes restart, service "X" might finally realize it needs correct credentials, but it's too late. Nodes are already expecting authorized requests, and this mismatch causes "X" to stop working on auth enabled and restarted nodes.
  • Infrequent traffic:  Suppose one of the services only interacts with the server once a week. Suppose it starts sending requests with incorrect credentials after authentication is enabled. Since the entire cluster is now running on authentication, the service's outdated credentials cause it to be denied access, resulting in a service-wide outage.

The overall aim of the proposed feature flag would allow clients to connect momentarily without authentication during the rollout, mitigating these risks and ensuring a smoother transition.

Thanks in advance for your continued review of the proposal.

 

On Mon, Feb 12, 2024 at 2:24 PM Abe Ratnofsky <a...@aber.io> wrote:
Hey Guarav,

Thanks for your proposal.

> disruptive, full-cluster restart, posing significant risks in live environments

For configuration that isn't hot-reloadable, like providing a new IAuthenticator implementation, a rolling restart is required. But rolling restarts are zero-downtime and safe in production, as long as you pace them accordingly.

In general, changing authenticators is a risky thing because it requires coordination with clients. To mitigate this risk and support clients while they transition between authenticators, I like the approach taken by MutualTlsWithPasswordFallbackAuthenticator:

If client certificates are available, then use those, otherwise use the existing PasswordAuthenticator that clients are already using. The existing IAuthenticator interface supports this transitional behavior well.

Your proposal to include a new configuration for auth_enforcement_flag doesn't clearly cover how to transition from one authenticator to another. It says:

> Soft: Operates in a monitoring mode without enforcing authentication

Most users use authentication today, so auth_enforcement_flag=Soft would allow unauthenticated clients to connect to the database.

--
Abe

On Feb 12, 2024, at 2:44 PM, Gaurav Agarwal <gaurav.iit...@gmail.com> wrote:

Dear Cassandra Community,

I'm excited to share a proposal for a new feature that I believe would significantly enhance the platform's security and operational flexibility: a flexible authentication mechanism implemented through a feature flag .

Currently, enforcing authentication in Cassandra requires a disruptive, full-cluster restart, posing significant risks in live environments. My proposal, the auth_enforcement_flag, addresses this challenge by offering three modes:

Hard: Enforces strict authentication with detailed logging.
Soft: Monitors connection attempts (valid and invalid) without enforcing authentication.
None: Maintains the current Cassandra behavior.

This flag enables:
Minimized downtime: Seamless authentication rollout without service interruptions.
Enhanced security: Detailed logs for improved threat detection and troubleshooting.
Gradual adoption: Phased implementation with real-world feedback integration.

I believe this feature provides substantial benefits for both users and administrators. Please see the detailed proposal here: Introducing flexible authentication mechanism

I warmly invite the community to review this proposal and share your valuable feedback. I'm eager to discuss its potential impact and collaborate on making Cassandra even better.

Thank you for your time and consideration.

Sincerely,
Gaurav Agarwal
Software Engineer at Uber

Reply via email to