Some interesting updates on current developments Open Assistant is open source ChatGPT clone with crowdsourced fine-tuning - https://open-assistant.io
Redpajama is project for reproducing LLaMA and releasing the model under open source licence. Current status is that they have released the pre-training data - https://www.together.xyz/blog/redpajama Free-dolly is CC-BY-SA licenced fine-tuning dataset. - https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm LLM chat which runs in web browser (non-open source Vicuna-7B) - https://simonwillison.net/2023/Apr/16/web-llm/ Afaik all of these are clear steps towards full open source LLM stack. Especially Open Assistant is especially interesting as it is focusing on crowdsourcing. Br, -- Kimmo Virtanen, Zache On Mon, Apr 3, 2023 at 8:43 PM Samuel Klein <meta...@gmail.com> wrote: > At this point I guess I would recommend adding five or so >> g2.cores8.ram36.disk20 flavor VPSs to WMCS, with between one and three >> RTX A6000 GPUs each, plus a 1TB SSD each, which should cost under >> $60k. That should allow for very widely multilingual models somewhere >> between GPT-3.5 and 4 performance with current training rates. >> > > Having part of the cluster for this makes sense, even as what it is used > for changes over time. > > >> These models can be quantized into int4 weights which run on cell >> phones: >> https://github.com/rupeshs/alpaca.cpp/tree/linux-android-build-support >> It seems inevitable that we will someday include such LLMs with >> Internet-in-a-Box, and, why not also the primary mobile apps >> > > Eventually, yes. A good reason to renew attention to mobile as a canonical > wiki experience. > > > _______________________________________________ > Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines > at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and > https://meta.wikimedia.org/wiki/Wikimedia-l > Public archives at > https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/R2KSXD3VE4MPGSIIYCPUMFXFPIY7D5CH/ > To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org
_______________________________________________ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/VEKGOASMRNUN4ZJSVRMJNBTRTH2VHWCA/ To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org
