I have been exploring how effective RAG-focused LLMs can be in accurately answering questions concerning the software architecture of various popular open source projects. Kafka is one of those projects. With over a decade of experience as a software architect, I have formed some opinions about what constitutes a good signal when analyzing large amounts of code for this purpose. So, I wrote this web application that allows you to ask these kinds of questions. You can ask about the current architecture or about how to enhance that architecture to accommodate new capabilities or to pay down specific forms of tech debt.

It seems to me that the best way to measure accuracy would be for those folks who work on Kafka to ask some questions and review the answers. Would anyone here be willing to participating?

Reply via email to