Mistral (LLM)

Run an Opensource LLM!

CLI Example Command

lilypad run mistral-7b-instruct:v0.1-lilypad3 -i PromptEnv="PROMPT=Name 10 renaissance artists"

Output Example in stout

1. Leonardo da Vinci

2. Michelangelo

3. Raphael

4. Titian

5. Albrecht Dürer

6. Sandro Botticelli

7. Jan van Eyck

8. Leon Battista Alberti

9. Johannes Gutenberg

10. Andrea Mantegna

Github Module

https://github.com/bacalhau-project/lilypad-module-mistral-7b-instruct/blob/main/lilypad_module.json.tmpl

{
    "machine": {
        "gpu": 1,
        "cpu": 1000,
        "ram": 1000
    },
    "job": {
        "APIVersion": "V1beta1",
        "Metadata": {
            "CreatedAt": "0001-01-01T00:00:00Z",
            "Requester": {}
        },
        "Spec": {
            "Deal": {
                "Concurrency": 1
            },
            "Docker": {
                "Entrypoint": [
                    "bash", "-c",
                    "python3 inference.py 2>/dev/null"
                ],
                "Image": "quay.io/lukemarsden/axolotl:v0.0.2",
                "EnvironmentVariables": [
                    {{if .PromptEnv}}{{.PromptEnv}}{{else}}"PROMPT=say give me a prompt"{{end}},
                    {{if .SeedEnv}}{{.SeedEnv}}{{else}}"RANDOM_SEED=42"{{end}},
                    "OUTPUT_DIR=/outputs/",
                    "HF_HUB_OFFLINE=1"
                ]
            },
            "Engine": "Docker",
            "Language": {
                "JobContext": {}
            },
            "Network": {
                "Type": "None"
            },
            "PublisherSpec": {
                "Type": "Estuary"
            },
            "Resources": {
                "GPU": "1"
            },
            "Timeout": 1800,
            "Verifier": "Noop",
            "Wasm": {
                "EntryModule": {}
            },
            "outputs": [
                {
                    "Name": "outputs",
                    "StorageSource": "IPFS",
                    "path": "/outputs"
                }
            ]
        }
    }
}