sungwy commented on PR #16538: URL: https://github.com/apache/iceberg/pull/16538#issuecomment-4559946620
> I'm good with this as is, I assume we'll be iterating on this in the future. Could you do a quick analysis of all of the issues we recently raised (check for rdblue's recently created issues) and give us a summary of what the threat model says about each issue? Thanks @RussellSpitzer Yeah, I initially ran Codex/GPT-5 against this model which had a 75-80% accuracy in correctly classifying those reports. Claude Opus 4.7 did even better at around 85%. I've just uploaded the detailed summary of these runs in this [public gist](https://gist.github.com/sungwy/d9ef24d4e9939c1c8c23f81b44450767). In short, in the Opus run, 7 reports out of 43 required human review. Of these 1 was a valid security report and 2 others were handled in _Private_ first, not as Iceberg security disclosures, but to give catalog maintainers a heads-up before public disclosure. The other 4 reports are, in my opinion, reports that would be best to classify as ambiguous reports that would be more safe to involve human review for now. So I agree with iterating on this over time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
