[jira] [Comment Edited] (CASSANDRA-15076) Align record header of FQL and audit binary log
[
https://issues.apache.org/jira/browse/CASSANDRA-15076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984231#comment-16984231
]
Marcus Eriksson edited comment on CASSANDRA-15076 at 11/28/19 8:36 AM:
---
-[~eperott] was preparing to commit this, but noticed
[this|https://github.com/apache/cassandra/pull/372/files#diff-bdaab1104a93e723ce0b609a6477c9c4L1262-L1266]
- was that included on purpose?-
edit: yep, leftover from CASSANDRA-14373
was (Author: krummas):
[~eperott] was preparing to commit this, but noticed
[this|https://github.com/apache/cassandra/pull/372/files#diff-bdaab1104a93e723ce0b609a6477c9c4L1262-L1266]
- was that included on purpose?
> Align record header of FQL and audit binary log
> ---
>
> Key: CASSANDRA-15076
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15076
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Tools, Tool/fql
>Reporter: Per Otterström
>Assignee: Per Otterström
>Priority: Normal
> Fix For: 4.0
>
>
> The new full query logger and the audit logger support logging into binary
> Chronicle logs. Both create records with a small header to indicate what
> follows, but the two features have adopted different header formats. Let's
> align the record header format with this ticket.
> * Both features should use the same header layout. This makes it possible to
> give more user friendly error messages in the {{fqltool}} and
> {{auditlogviewr}} commands.
> * The record header should have a distinct {{type}} to indicate the type of
> record.
> * The record header should have a {{version}} so that the record format can
> evolve.
> Current record header format of the FQL is:
> {noformat}
> version:0(int16)
> type:(text)
> {noformat}
> where {{}} can be either {{batch}} or {{single-query}}.
> Current record header format of the binary audit log is:
> {noformat}
> type:AuditLog(text)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15076) Align record header of FQL and audit binary log
[
https://issues.apache.org/jira/browse/CASSANDRA-15076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950744#comment-16950744
]
Vinay Chella edited comment on CASSANDRA-15076 at 10/14/19 6:28 AM:
First pass of review:
* {{AuditLogViewer::verifyVersion}} and {{AuditLogViewer::readType}} are
throwing runtime exceptions in case of unexpected format and unsupported
versions. However, when reading log files with thousands of records, earlier
schematics were to ignore those records. Are we stopping the processing of the
audit log in new schematics? If so, It might be better to log warnings for
those specific records (_if needed)_ and move on with processing rather than
aborting the viewer.
* nitpick/idea: Would {{int16}} be the right choice for version? Can we be
aggressive to {{int8}}?
Can you also rebase your changes on the latest trunk?
was (Author: vinaykumarcse):
First pass of review:
* {{AuditLogViewer::verifyVersion}} and {{AuditLogViewer::readType}} are
throwing runtime exceptions in case of unexpected format and unsupported
versions. When reading log files with thousands of records, earlier schematics
were to ignore those records, but with new schematics, are we stopping the
processing of the audit log? If so, It might be better to log warnings for
those specific records (_if needed)_ and move on rather than aborting the
viewer.
* nitpick/idea: Would {{int16}} be the right choice for version? Can we be
aggressive to {{int8}}?
Can you also rebase your changes on the latest trunk?
> Align record header of FQL and audit binary log
> ---
>
> Key: CASSANDRA-15076
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15076
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Tools, Tool/fql
>Reporter: Per Otterström
>Assignee: Per Otterström
>Priority: Normal
> Fix For: 4.0
>
>
> The new full query logger and the audit logger support logging into binary
> Chronicle logs. Both create records with a small header to indicate what
> follows, but the two features have adopted different header formats. Let's
> align the record header format with this ticket.
> * Both features should use the same header layout. This makes it possible to
> give more user friendly error messages in the {{fqltool}} and
> {{auditlogviewr}} commands.
> * The record header should have a distinct {{type}} to indicate the type of
> record.
> * The record header should have a {{version}} so that the record format can
> evolve.
> Current record header format of the FQL is:
> {noformat}
> version:0(int16)
> type:(text)
> {noformat}
> where {{}} can be either {{batch}} or {{single-query}}.
> Current record header format of the binary audit log is:
> {noformat}
> type:AuditLog(text)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15076) Align record header of FQL and audit binary log
[
https://issues.apache.org/jira/browse/CASSANDRA-15076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950744#comment-16950744
]
Vinay Chella edited comment on CASSANDRA-15076 at 10/14/19 6:26 AM:
First pass of review:
* {{AuditLogViewer::verifyVersion}} and {{AuditLogViewer::readType}} are
throwing runtime exceptions in case of unexpected format and unsupported
versions. When reading log files with thousands of records, earlier schematics
were to ignore those records, but with new schematics, are we stopping the
processing of the audit log? If so, It might be better to log warnings for
those if needed and move on rather than aborting the viewer.
* nitpick/idea: Would {{int16}} be the right choice for version? Can we be
aggressive to {{int8}}?
Can you also rebase your changes on the latest trunk?
was (Author: vinaykumarcse):
First pass of review:
* {{AuditLogViewer::verifyVersion}} and {{AuditLogViewer::readType}} are
throwing runtime exceptions in case of unexpected format and unsupported
versions. When reading log files with thousands of records, earlier schematics
were to ignore those records, but with new schematics, are stopping the
processing of the audit log? If so, It might be better to log warnings for
those if needed and move on rather than aborting the viewer.
* nitpick/idea: Would {{int16}} be the right choice for version? Can we be
aggressive to {{int8}}?
Can you also rebase your changes on the latest trunk?
> Align record header of FQL and audit binary log
> ---
>
> Key: CASSANDRA-15076
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15076
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Tools, Tool/fql
>Reporter: Per Otterström
>Assignee: Per Otterström
>Priority: Normal
> Fix For: 4.0
>
>
> The new full query logger and the audit logger support logging into binary
> Chronicle logs. Both create records with a small header to indicate what
> follows, but the two features have adopted different header formats. Let's
> align the record header format with this ticket.
> * Both features should use the same header layout. This makes it possible to
> give more user friendly error messages in the {{fqltool}} and
> {{auditlogviewr}} commands.
> * The record header should have a distinct {{type}} to indicate the type of
> record.
> * The record header should have a {{version}} so that the record format can
> evolve.
> Current record header format of the FQL is:
> {noformat}
> version:0(int16)
> type:(text)
> {noformat}
> where {{}} can be either {{batch}} or {{single-query}}.
> Current record header format of the binary audit log is:
> {noformat}
> type:AuditLog(text)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15076) Align record header of FQL and audit binary log
[
https://issues.apache.org/jira/browse/CASSANDRA-15076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950744#comment-16950744
]
Vinay Chella edited comment on CASSANDRA-15076 at 10/14/19 6:26 AM:
First pass of review:
* {{AuditLogViewer::verifyVersion}} and {{AuditLogViewer::readType}} are
throwing runtime exceptions in case of unexpected format and unsupported
versions. When reading log files with thousands of records, earlier schematics
were to ignore those records, but with new schematics, are we stopping the
processing of the audit log? If so, It might be better to log warnings for
those specific records (_if needed)_ and move on rather than aborting the
viewer.
* nitpick/idea: Would {{int16}} be the right choice for version? Can we be
aggressive to {{int8}}?
Can you also rebase your changes on the latest trunk?
was (Author: vinaykumarcse):
First pass of review:
* {{AuditLogViewer::verifyVersion}} and {{AuditLogViewer::readType}} are
throwing runtime exceptions in case of unexpected format and unsupported
versions. When reading log files with thousands of records, earlier schematics
were to ignore those records, but with new schematics, are we stopping the
processing of the audit log? If so, It might be better to log warnings for
those if needed and move on rather than aborting the viewer.
* nitpick/idea: Would {{int16}} be the right choice for version? Can we be
aggressive to {{int8}}?
Can you also rebase your changes on the latest trunk?
> Align record header of FQL and audit binary log
> ---
>
> Key: CASSANDRA-15076
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15076
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Tools, Tool/fql
>Reporter: Per Otterström
>Assignee: Per Otterström
>Priority: Normal
> Fix For: 4.0
>
>
> The new full query logger and the audit logger support logging into binary
> Chronicle logs. Both create records with a small header to indicate what
> follows, but the two features have adopted different header formats. Let's
> align the record header format with this ticket.
> * Both features should use the same header layout. This makes it possible to
> give more user friendly error messages in the {{fqltool}} and
> {{auditlogviewr}} commands.
> * The record header should have a distinct {{type}} to indicate the type of
> record.
> * The record header should have a {{version}} so that the record format can
> evolve.
> Current record header format of the FQL is:
> {noformat}
> version:0(int16)
> type:(text)
> {noformat}
> where {{}} can be either {{batch}} or {{single-query}}.
> Current record header format of the binary audit log is:
> {noformat}
> type:AuditLog(text)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15076) Align record header of FQL and audit binary log
[
https://issues.apache.org/jira/browse/CASSANDRA-15076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816085#comment-16816085
]
Per Otterström edited comment on CASSANDRA-15076 at 4/12/19 8:52 AM:
-
Sounds reasonable to me.
So FQL header remain as is. Audit record header will be:
{noformat}
version:0(int16)
type:audit(text)
{noformat}
And we'll update {{fqltool}} and {{auditlogviewr}} to give useful error
messages when an unexpected header version or type is detected.
I'd be willing to work on this ticket either as assignee or reviewer.
[~vinaykumarcse], would you be able to work with me on this?
I'm about to cut a new release of [ecAudit|https://github.com/Ericsson/ecaudit]
soon where support for a Chronicle backend is added. Having an agreement on
record header format in this ticket would simplify upgrades from Cassandra 3.x
+ ecAudit -> Cassandra 4.0.
was (Author: eperott):
Sounds reasonable to me.
So FQL header remain as is. Audit record header will be:
{noformat}
type:audit(text)
version:0(int16)
{noformat}
And we'll update {{fqltool}} and {{auditlogviewr}} to give useful error
messages when an unexpected header version or type is detected.
I'd be willing to work on this ticket either as assignee or reviewer.
[~vinaykumarcse], would you be able to work with me on this?
I'm about to cut a new release of [ecAudit|https://github.com/Ericsson/ecaudit]
soon where support for a Chronicle backend is added. Having an agreement on
record header format in this ticket would simplify upgrades from Cassandra 3.x
+ ecAudit -> Cassandra 4.0.
> Align record header of FQL and audit binary log
> ---
>
> Key: CASSANDRA-15076
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15076
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Tools, Tool/fql
>Reporter: Per Otterström
>Priority: Normal
> Fix For: 4.0
>
>
> The new full query logger and the audit logger support logging into binary
> Chronicle logs. Both create records with a small header to indicate what
> follows, but the two features have adopted different header formats. Let's
> align the record header format with this ticket.
> * Both features should use the same header layout. This makes it possible to
> give more user friendly error messages in the {{fqltool}} and
> {{auditlogviewr}} commands.
> * The record header should have a distinct {{type}} to indicate the type of
> record.
> * The record header should have a {{version}} so that the record format can
> evolve.
> Current record header format of the FQL is:
> {noformat}
> version:0(int16)
> type:(text)
> {noformat}
> where {{}} can be either {{batch}} or {{single-query}}.
> Current record header format of the binary audit log is:
> {noformat}
> type:AuditLog(text)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
