michaelsembwever commented on code in PR #2062: URL: https://github.com/apache/cassandra/pull/2062#discussion_r1087649059
########## doc/modules/cassandra/pages/operating/auditlogging.adoc: ########## @@ -186,52 +210,101 @@ excluded_categories: DDL, DML, QUERY, PREPARE Users to audit log are set with the `included_users` and `excluded_users` options. The `included_users` option specifies a comma-separated list of users to include explicitly. The `excluded_users` option specifies a comma-separated list of users to exclude explicitly. -By default all users are included, and no users are excluded. +By default, all users are included, and no users are excluded. [source, yaml] ---- included_users: excluded_users: john, mary ---- +[[roll_cycle]] === roll_cycle The ``roll_cycle`` defines the frequency with which the audit log segments are rolled. -Supported values are ``HOURLY`` (default), ``MINUTELY``, and ``DAILY``. +Supported values are: + +- ``MINUTELY`` +- ``FIVE_MINUTELY`` +- ``TEN_MINUTELY`` +- ``TWENTY_MINUTELY`` +- ``HALF_HOURLY`` +- ``HOURLY`` (default) +- ``TWO_HOURLY`` +- ``FOUR_HOURLY`` +- ``SIX_HOURLY`` +- ``DAILY`` + For example: ``roll_cycle: DAILY`` +WARNING: Read the following paragraph when changing ``roll_cycle`` on a production node. + +With the `BinLogger` implementation, any attempt to modify the roll cycle on a node where audit logging was previously enabled will fail silentely due to https://github.com/OpenHFT/Chronicle-Queue[Chronicle Queue] roll cycle inference mechanism (even if you delete the ``metadata.cq4t`` file). + +Here is an example of such an override visible in Cassandra logs: +---- +INFO [main] <DATE TIME> BinLog.java:420 - Attempting to configure bin log: Path: /path/to/audit Roll cycle: TWO_HOURLY [...] +WARN [main] <DATE TIME> SingleChronicleQueueBuilder.java:477 - Overriding roll cycle from TWO_HOURLY to FIVE_MINUTE +---- + +In order to change ``roll_cycle`` on a node, you have to: + +1. Stop Cassandra +2. Move or offload all audit logs somewhere else (in a safe and durable location) +3. Restart Cassandra. +4. Check Cassandra logs +5. Make sure that audit log filenames under ``audit_logs_dir`` correspond to the new roll cycle. + === block The ``block`` option specifies whether audit logging should block writing or drop log records if the audit logging falls behind. Supported boolean values are ``true`` (default) or ``false``. -For example: ``block: false`` to drop records + +For example: ``block: false`` to drop records (e.g. if audit is used for troobleshooting) + +For regulatory compliance purpose, it's a good practice to explicitely set ``block: true`` to prevent any regression in case of future default value change. === max_queue_weight The ``max_queue_weight`` option sets the maximum weight of in-memory queue for records waiting to be written to the file before blocking or dropping. The option must be set to a positive value. The default value is 268435456, or 256 MiB. + For example, to change the default: ``max_queue_weight: 134217728 # 128 MiB`` === max_log_size The ``max_log_size`` option sets the maximum size of the rolled files to retain on disk before deleting the oldest file. The option must be set to a positive value. The default is 17179869184, or 16 GiB. For example, to change the default: ``max_log_size: 34359738368 # 32 GiB`` +WARNING: ``max_log_size`` is ignored if ``archive_command`` option is set. + +[[archive_command]] === archive_command +NOTE: If ``archive_command`` option is empty or unset (default), Cassandra uses a built-in DeletingArchiver that deletes the oldest files if ``max_log_size`` is reached. + The ``archive_command`` option sets the user-defined archive script to execute on rolled log files. -For example: ``archive_command: /usr/local/bin/archiveit.sh %path # %path is the file being rolled`` +For example: ``archive_command: "/usr/local/bin/archiveit.sh %path"`` -=== max_archive_retries +``%path`` is replaced with the absolute file path of the file being rolled. -The ``max_archive_retries`` option sets the max number of retries of failed archive commands. The default is 10. -For example: ``max_archive_retries: 10`` +When using a user-defined script, Cassandra do **not** use the DeletingArchiver, so it's the responsability of the script to make any required cleanup. +Cassandra will call the user-defined script as soon as the log file is rolled. It means that Chronicle Queue's QueueFileShrinkManager will not be able to shrink the sparse log file because it's done asynchronously. In other words, all log files will have at least the size of the default block size (80 MiB), even if there are only a few KB of real data. Consequently, some warnings will appear in Cassandra system.log: -An audit log file could get rolled for other reasons as well such as a -log file reaches the configured size threshold. +---- +WARN [main/queue~file~shrink~daemon] <DATE TIME> QueueFileShrinkManager.java:63 - Failed to shrink file as it exists no longer, file=/path/to/xxx.cq4 +---- -Audit logging can also be configured using ``nodetool` when enabling the feature, and will override any values set in the `cassandra.yaml` file, as discussed in the next section. +TIP: Because Cassandra does not make use of Pretoucher, you can configure Chronicle Queue to shrink files synchronously -- i.e. as soon as the file is rolled -- with ``chronicle.queue.synchronousFileShrinking`` JVM properties. For instance, you can add the following line at the end of ``cassandra-env.sh``: ``JVM_OPTS="$JVM_OPTS -Dchronicle.queue.synchronousFileShrinking=true"`` Review Comment: Does it render correctly when generated? (that really is all we care about – i suspect thinking IDEA rendering helps traps us) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
