On 12/30/23 21:40, Mo Zhou wrote:
I am not
able to develop DebGPT and confess I am not investing my time in
learning to do it. But can we attract the people who want to tinker in
this direction?
Debian funds should be able to cover the hardware requirement and
training expenses even if they are slightly expensive. The more
expensive thing is the time of domain experts. I can train such a
model but clearly I do not have bandwidth for that.
No. I changed my mind.
I can actually quickly wrap some debian-specific prompts with an
existing chatting LLM. This is easy and does not need expensive hardware
(although it may still require 1~2 GPUs with 24GB memory for inference),
nor any training procedure.
The project repo is created here
https://salsa.debian.org/deeplearning-team/debgpt
I have enabled issues. And maybe people interested in this can redirect
the detailed discussions to the repo issues.
I'm sure it is already possible to let LLM read the long policy
document, or debhelper man pages for us, and provide some suggestions or
patches. The things I'm uncertain is (1) how well a smaller LLM, like 7B
or 13B ones can do compared to proprietary LLMs in this case; (2) how
well a smaller LLM can be when it is quantized to int8 or even int4 for
laptops.
Oh, BTW, the dependencies needed by the project are not complete in
debian archive.
