llama.cpp
#########

.. seealso:: https://github.com/ggml-org/llama.cpp

Téléchargement des modéles
**************************

.. seealso:: https://huggingface.co/ggml-org

Les modéles *7B* fonctionne sur la 9070XT (16GB RAM) mais seulement 1 par 1, pour avoir 1
coder + 1 instruct j'utilise les *3B*.

.. code-block:: console

   curl -JOL --output-dir ~/.local/share/llama/models https://huggingface.co/ggml-org/Qwen2.5-Coder-3B-Q8_0-GGUF/resolve/main/qwen2.5-coder-3b-q8_0.gguf
   curl -JOL --output-dir ~/.local/share/llama/models https://huggingface.co/ggml-org/Qwen2.5-Coder-3B-Instruct-Q8_0-GGUF/resolve/main/qwen2.5-coder-3b-instruct-q8_0.gguf
   #curl -JOL --output-dir ~/.local/share/llama/models https://huggingface.co/ggml-org/Qwen2.5-Coder-7B-Q8_0-GGUF/resolve/main/qwen2.5-coder-7b-q8_0.gguf
   #curl -JOL  --output-dir ~/.local/share/llama/models https://huggingface.co/ggml-org/Qwen2.5-Coder-7B-Instruct-Q8_0-GGUF/resolve/main/qwen2.5-coder-7b-instruct-q8_0.gguf

Service
*******

.. literalinclude:: llama.cpp/llama.service
   :language: ini
   :caption: ~/.config/systemd/user/llama.service

.. code-block:: console

   systemctl --user daemon-reload
   systemctl --user start llama.service