Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Get Started with Lilypad v0 - Call off-chain distributed compute from your smart contract!
Lilypad v0 Integrated Networks
The Lilypad v0 Smart Contracts
Creating Bacalhau Job Spec's
Lilypad v0 is a "bridge" for running Bacalhau compute jobs via a smart contracts.
Running a Stable Diffusion Job from a smart contract with Lilypad v0
Implementing an unbiased off-chain voting process using Lilypad and Bacalhau, with Drand Time-lock encryption
Running a fraud detection job with Lilypad
Trustless Distributed Compute for web3
Ideas and Guides for Contributing to Lilypad
Current State
A cowsay job
Fine-tuning models from inputs.
A Fast Chat LLM Inference Module for Lilypad
Community Member Contribution
You can use Lilypad to run AI workload models including Stable Diffusion and Stable Diffusion Video, or you can add your own module to run on the Lilypad Network. Using Lilypad's distributed compute nodes, you can build and run your own containerized workloads that require high-performance computing.
Obtaining
is developing a serverless, distributed compute network that enables internet-scale data processing, AI, ML & other arbitrary computation, while unleashing idle processing power & unlocking a new marketplace for compute.
A Lilypad Job Module is a standard containerized (Docker) process for running compute workloads on Lilypad. Easily deploy an AI model or other compute intensive workloads to the Lilypad decentralized GPU network. Learn more about running AI workloads via , and use our or to start sending job requests.
Build your own Lilypad job modules either or with our package.
The Lilypad Network aggregates idle compute (CPU and GPU) capacity from providers around the world. Compute providers on the network are referred to as .
If you're interested in becoming a Lilypad Resource Provider, first make sure that you meet our , then register for the . Learn more about running a Resource Provider on Lilypad !
There are many ways to for participating in the Lilypad economy! Resource Providers earn rewards for running compute jobs and the community earns rewards for building on the network and supporting the ecosystem! Find all rewards opportunities below:
Apply to the RP Beta program and earn Resource Providers
Join the by submitting info
Add a model to run on Lilypad or build a project listed in the
Complete open source
and start contributing!
View the .
Configure a crypto wallet to receive testnet tokens used to interact with the Lilypad Network
The wallet you use for your account must have both ETH (to run smart contracts on Ethereum) and Lilypad (LP) tokens in order to pay for jobs (or receive funds for jobs) on the network.
The Lilypad Testnet (IncentiveNet) is currently running on the Arbitrum L2 network built on Ethereum.
In order to change to the Arbitrum network in the wallet, open MetaMask and click the network button in the top left of the menu bar:
Then select "Add network":
Next, select "Add a network manually":
Input the required Arbitrum Sepolia Testnet network info, and then "Save":
Network name: Arbitrum Sepolia
Chain ID: 421614
Currency symbol: ETH
The wallet is now setup and will display an ETH (Arbitrum Sepolia) token balance. In order to also display the LP token balance, the LP token will need to be imported.
Select "Import tokens" from the three dot menu next to the network name:
Select "Custom token" and add the Lilypad token contract address and token symbol. Then "Save".
Token contract address: 0x0352485f8a3cB6d305875FaC0C40ef01e0C06535
Token symbol: LP
You should now see both ETH and LP listed in the wallet (initial ETH and LP balances will be 0).
Now you're ready to fund the wallet with testnet LP and ETH tokens!
Get Testnet Lilypad Tokens (LP) and Arbitrum Sepolia Testnet ETH
tldr: To obtain funds, first ensure the wallet is connected to the Arbitrum Sepolia network. then, collect LP and ETH tokens from these faucets:
Follow these steps to successfully claim your Testnet LP tokens:
Authenticate with Discord.
Copy your MetaMask wallet address into the input.
Click "Request".
Testnet LP tokens will be sent to the wallet address provided to the faucet.
With a balance of both LP and ETH, you're ready to run jobs with the Lilypad CLI!
the Lilypad CLI.
.
, the Lilypad module equivalent to Hello, World!
Both Resource Providers (GPU compute nodes) and those looking to run jobs on the Lilypad network need to set up a Metamask account in order to run jobs on Lilypad. The public key of your wallet address is how you are identified on the network, and is how you can look up the transactions you make on the . You need an account for running jobs, and a separate account for each GPU you want to set up on the network.
End users of Lilypad can decide which crypto wallet they would like to use. In this guide, we advise using a crypto wallet.
Install
Network info is referenced directly from the Arbitrum Sepolia .
New RPC URL:
Block explorer URL: (optional)
(3rd party faucet list)
Find out why you need tokens in the
You must be a member of to claim tokens
Navigate to the Lilypad Testnet .
Get Arbitrum Sepolia ETH from this list of . Each faucet is designed differently, so follow the instructions provided.
If you need more tokens and already have Sepolia ETH, use the to transfer the tokens over to Arbitrum Sepolia.
Instructions on how to use the Lilypad API
Once you have your API key, you can start running AI inference jobs using Lilypad's Anura API. Below is a simple "Hello World" example to get started.
Before running a job, check which models are supported:
Use the API to send a simple chat completion request using one of the available models:
Replace "MODEL_NAME:MODEL_VERSION" with the model you want to use from the previous step.
You can check the status of your job and retrieve outputs using the job ID:
This will return details on whether the job is still processing or has completed.
Instructions for getting started with the Lilypad API
To interact with Lilypad's API, you'll first need to create an account and obtain an API key for authentication. This key allows you to submit jobs and retrieve results securely.
Next, log in to the dashboard and click "Create API key".
Now that you've created a new API key you can copy and securely store your API key for usage.
An open, ongoing program designed to help you build with Lilypad. There are no deadlines or fixed formats. Whether you're creating modules, building agent workflows or integrating Lilypad into a larger stack, Builder-verse gives you the space, support and rewards to grow your ideas.
Lilypad rewards contributors who help grow the network, whether you're building modules and agents or providing GPU compute.
Building agents or full-stack applications that use Lilypad modules
Driving usage. The more your module or agent is used, the more rewards you earn
Your own custom AI modules
Agent pipelines using existing modules
Apps using Lilypad’s CLI or API
Integrations with your existing tools or workflows
Looking for inspiration or guidance? Check out these resources to get you started!
Join the Lilypad Discord and follow us on socials to stay up to date with the action!
Running the most important Hello World on Lilypad!
Information about the Lilybit_ rewards program
There are multiple ways to participate in the Lilypad ecosystem and earn rewards for your contributions.
In exchange for providing compute power to Lilypad's decentralized compute network, Resource Providers (RPs) accumulate Lilybit_ credits based on their contributions to the network.
RP incentives on Testnet will be provided via the Lilypad RP Beta Program. Any RP can submit info to join the RP beta program, although RPs with more resources will be prioritized in order to run large models on Lilypad Network.
Perform off-chain decentralized compute over data, with on-chain guarantees. Call this functionality directly from a smart contract, CLI, and an easy to use abstraction layer.
The network is actively laying groundwork for multi-chain integration and the deployment of an incentivized testnet.
Lilypad has evolved from earlier versions (v0, v1 & v2), where the network served as a proof of concept for verifiable, decentralized compute directly from smart contracts. These earlier iterations established the groundwork for what is now a robust, scalable platform with expanded features and multichain support.
Bacalhau has been integral to Lilypad since its early versions (v0 and v1), serving as the backbone for verifiable off-chain compute. In these early iterations, Bacalhau was used for Proof of Concept projects, helping users execute decentralized compute jobs from smart contracts.
Lilypad aims to mitigate the challenges predominantly associated with the accessibility of high-performance computational hardware. At present, numerous barriers impede developers and organizations from smoothly integrating projects that require high-performance computing, such as AI technologies, into their applications.
Unlike conventional centralized systems, where access to powerful compute hardware is restricted and costly, Lilypad endeavors to democratize this access. Through its verifiable, trustless, and decentralized computational network, Lilypad extends unrestricted, global access to computational power. By leveraging decentralized infrastructure networks such as Filecoin, Lilypad is strategically positioned to enhance the efficiency, transparency, and accessibility of high-performance computing hardware.
Perform off-chain decentralized compute over data, with on-chain guarantees, and to call this functionality directly from a smart contract, CLI, API and an easy to use abstraction layer, opens the door to a multitude of possible applications including:
Inference AI jobs
ML training jobs
Invoking & supporting generic ZK computations
Cross-chain interoperability complement to bridge protocols
Federated Learning consensus (with Bacalhau insulated jobs)
IOT & Sensor Data integrations
Providing a platform for Digital twins
Supply chain tracking & analysis
ETL & data preparation jobs
Some of the key features of Lilypad include:
Verifiable Serverless Decentralized Compute Network: Lilypad is a decentralized compute network that aims to provide global, permissionless access to compute power. The Network orchestrates off-chain compute (a global GPU marketplace) and uses on-chain verification (Arbitrum L2 on Ethereum) to provide guarantees of compute success.
Open Compute Network: Lilypad is an open compute network allowing users to access and run AI models/other programs in a serverless manner. Module creators and general users can access a curated set of modules or can easily create their own Lilypad module to run an AI model/other program on the network.
Decentralization of Mediators: The team also aims to decentralize the mediators in the network. This means that the decision-making process and governance of the network will be distributed among multiple participants, ensuring a more decentralized and resilient system.
Instructions for installing the Lilypad CLI on your machine
The installation process for the Lilypad CLI involves several automated steps to configure it for your specific system. Initially, the setup script identifies your computer's architecture and operating system to ensure compatibility. It will then download the latest production build of the Lilypad CLI directly from the official GitHub repository using curl and wget.
Once the CLI tool is downloaded, the script sets the necessary permissions to make the executable file runnable. It then moves the executable to a standard location in your system's path to allow it to be run from any terminal window.
Lilypad offers two distinct installation options to cater to different roles within the network:
One for Lilypad users who wish to run compute jobs on the Lilypad Network.
Another for resource providers who supply the computational resources to the Lilypad Network.
Select the appropriate installation based on your role:
You're required to set your private key environment variable, WEB3_PRIVATE_KEY, to interact securely with the network.
To verify the installation, running lilypad in the terminal should display a list of available commands, indicating that Lilypad CLI is ready to use.
To uninstall Lilypad, you'll need to remove the lilypad binary.
The following command detects where the lilypad binary is located and removes it.
🚨 Using
sudo rm -rfcan be dangerous if not used carefully. Proceed with caution.
You can verify Lilypad has been uninstalled successfully with the following command:
If the uninstall was successful, you should see the message lilypad not found.
For more advanced usage, including streaming completions, multi-turn conversations, and additional endpoints, check out the .
Visit the and sign up using an email address.
is a space for developers, labs, and builders to create powerful tools and workflows on the Lilypad Network. It’s part of our mission to accelerate open source AI through decentralized compute infrastructure and accessible tooling.
You don’t need to wait for a hackathon. Submit anytime and track your impact on the . Top contributors may also be eligible for additional support, amplification, and future grant opportunities.
Creating and
- Step-by-step guide to creating your own Lilypad module
- CLI tool to scaffold a new module with everything you need to get started
- Create composable AI workflows powered by Lilypad modules
Check out our for a curated collection of modules, templates, and community examples!
Before you run a Lilypad job, make sure you have and have set in your environment.
Your WEB3_PRIVATE_KEY can be retrieved from the MetaMask account details menu. For more info, check out the on how to get a your private key. Be sure to keep your private key safe and never share it or store it in unsecured places to prevent unauthorized access to your funds.
Lilypad provides distributed computational services underpinned by the . The network provides infrastructure for use cases like AI inference, ML training, DeSci and more. Lilypad strategically collaborates with decentralized infrastructure networks, such as Filecoin, to formulate a transparent, efficient, and accessible computational ecosystem.
Find Lilypad on or visit .
Mainstream Web3 Application Support: Lilypad is designed to enable mainstream web3 applications to use its compute network with the , and . It aims to make decentralized AI compute more accessible, efficient, and transparent for developers and users.
Multichain Support: The Lilypad Incentivized Testnet on Arbitrum in June 2024 with plans to go multi-chain in the near future. Supporting multiple blockchain networks will increase the scalability and interoperability of the network, allowing users to choose the blockchain that best suits their needs.
Incentivized Test Net: The is live! The IncentiveNet program provide users withto participate in running nodes, testing, and improving the network. Learn more by checking out the .
When using the CLI always ensure you're running the of Lilypad.
The resource provider version of Lilypad is not supported on Darwin/macOS.
A WEB3_PRIVATE_KEY can be retrieved from the Metamask account details menu. For more info, check out the on viewing a wallet's private key. Be sure to keep your private key safe and never share it or store it in unsecured places to prevent unauthorized access to your funds. You also need a separate private key
To use the Lilypad CLI, the set private key will need to hold Lilypad testnet tokens and Arbitrum Sepolia ETH. You can find those instructions in the documentation.
Thats it! You've successfully installed the Lilypad CLI on your machine!
Getting started with `create-lilypad-module`
Anura, Lilypad's official AI inference API
Use Anura to start running AI inference job modules on Lilypad's decentralized compute network:
Find which models we support:
Choose a model, customize your request and fire away:
Find which models we support:
Choose a model and generate your first image
Currently the rate limit for the api is set to 20 calls per second
To see which models are available:
POST /api/v1/chat/completions
Note: Due to the decentralized nature of the Lilypad Network we recommend using the streaming variant where possible at this time
This endpoint provides both a streaming interface using Server-Sent Events (SSE) and non-streaming interface for chat completions which is compliant with the OpenAI specification. This means that you can plug and play Anura using the OpenAI SDK by simply passing in the Anura Url and API Key into your client like so:
Request Headers
Content-Type: application/json*
Accept: text/event-stream (recommended for streaming)
Authorization: Bearer YOUR_API_KEY*
Request Parameters
model*
Model ID used to generate the response (e.g. deepseek-r1:7b). Required.
string
messages*
A list of messages comprising the conversation so far. Required.
array
Request Body (non-streaming)
Response Format (non-streaming)
The response is an OpenAI ChatCompletion Object with the following format:
Response Codes
200 OK: Request successful, stream begins
400 Bad Request: Invalid request parameters
401 Unauthorized: Invalid or missing API key
404 Not Found: Requested model not found
500 Internal Server Error: Server error processing request
Response Object Fields
The response data contains the following fields:
id
A unique identifier for the chat completion
object
The object type
created
Timestamp when the response was created
model
The model used for generation
choices
The array containing the assistant's response
choices[0].message.role
Always "assistant" for responses
choices[0].message.content
The generated text content
choices[0].message.tool_calls
The array containing the corresponding tool response objects (this is only applicable if you make a tool request)
choices[0].finish_reason
Reason for completion (e.g., "stop", "length")
usage.prompt_tokens
The number of tokens used in the prompt
usage.completion_tokens
The number of tokens in the generated completion
usage.total_tokens
The sum of the prompt_tokens and the completion_tokens
Request Body (streaming)
Response Format (streaming)
The response is a stream of Server-Sent Events (SSE) with chunked OpenAI ChatCompletion objects with the following format:
Initial response:
Processing updates:
Content delivery:
Completion marker:
Response Codes
200 OK: Request successful, stream begins
400 Bad Request: Invalid request parameters
401 Unauthorized: Invalid or missing API key
404 Not Found: Requested model not found
500 Internal Server Error: Server error processing request
Response Object Fields
The delta event data contains the following fields:
id
A unique identifier for the chat completion
object
The object type
created
Timestamp when the response was created
model
The model used for generation
choices
The array containing the assistant's response
choices[0].delta.role
Always "assistant" for responses
choices[0].delta.content
The generated text content
choices[0].delta.tool_calls
The array containing the corresponding tool response objects (this is only applicable if you make a tool request)
choices[0].finish_reason
Reason for completion (e.g., "stop", "length")
usage.prompt_tokens
The number of tokens used in the prompt
usage.completion_tokens
The number of tokens in the generated completion
usage.total_tokens
The sum of the prompt_tokens and the completion_tokens
Conversation Context
The API supports multi-turn conversations by including previous messages in the request:
This allows for contextual follow-up questions and maintaining conversation history.
The Anura chat completions endpoint supports requests with tooling allowing for function calling through many popular AI frameworks and sdks.
At the moment only a select number models support tooling including:
llama3.1:8b
qwen2.5:7b
qwen2.5-coder:7b
phi4-mini:3.8b
mistral:7b
Below is a sample request and response
Request:
Response:
The chat completions API also supports vision requests allowing for image-to-text search against a base64 encoded image. This will allow you to make a query against an image asking a LLM what the image is or about particular details around it. Currently vision is only supported via the following models (more coming soon):
llava:7b
gemma3:4b
Additionally, the vision capability is limited by the following constraints:
Images must only be base64 encoded (you cannot pass a link to an image at this time)
Maximum image size is 512px x 512px
Support for JPEG or PNG format
Request:
Response:
Use the embeddings endpoint to compute embeddings for user queries supported by the nomic-embed-text model. This endpoint is OpenAI compliant which means you can use it with the OpenAI SDK (see the end of the Embeddings section for a code example)
Endpoint
POST /api/v1/embeddings
Request Headers
Content-Type: application/json*
Authorization: Bearer YOUR_API_KEY*
Request Parameters
model*
Model ID used to generate the response (e.g. nomic-embed-text). Required.
string
input*
The input to create embeddings from. This can be either a single string or an array of strings. Required
string or array of strings
Request Sample (single input)
Response Sample (single input)
Request Sample (multiple input)
Response Sample (multiple input)
Response Codes
200 OK: Request successful, stream begins
400 Bad Request: Invalid request parameters
401 Unauthorized: Invalid or missing API key
404 Not Found: Requested model not found
500 Internal Server Error: Server error processing request
Example Code using the OpenAI SDK
The Anura API enables you to run stable diffusion jobs to generate images executed through our decentralized compute network. It's really easy to get started generating your own generative AI art using Anura through the endpoints we provide.
Retrieve the list supported image generation models
GET /api/v1/image/models
Request Headers
Content-Type: application/json*
Authorization: Bearer YOUR_API_KEY*
Request Parameters
model*
Model ID used to generate the response (e.g. sdxl-turbo). Required.
string
prompt*
The prompt input to generate your image from (max limit of 1000 characters)
string
Request Sample
Response
Response Codes
200 OK: Request successful, stream begins
400 Bad Request: Invalid request parameters
401 Unauthorized: Invalid or missing API key
404 Not Found: Requested model not found
500 Internal Server Error: Server error processing request
Currently we support sdxl-turbo; however, we are always adding new models, so stay tuned!
Generate an AI Image
POST /api/v1/image/generate
Request Headers
Content-Type: application/json*
Authorization: Bearer YOUR_API_KEY*
Request Parameters
model*
Model ID used to generate the response (e.g. sdxl-turbo). Required.
string
prompt*
The prompt input to generate your image from (max limit of 1000 characters)
string
Request Sample
Alternatively you can also make the same request through a curl command and have the image be output to a file on your machine
The result of running this command will be the creation of the spaceship.png file in the directory you ran the command from.
Response
This endpoint will return the raw bytes value of the image that was generated which you can output to a file (like shown in the curl command above) or place it in a buffer to write to a file in your app, e.g.
Note: Should you ever need to know what the corresponding Job Offer ID for image generation, it is provided in the response header as Job-Offer-Id
Response Codes
200 OK: Request successful, stream begins
400 Bad Request: Invalid request parameters
401 Unauthorized: Invalid or missing API key
404 Not Found: Requested model not found
500 Internal Server Error: Server error processing request
The Anura API enables you to run long running jobs to generate videos executed through our decentralized compute network. It's really easy to get started generating your own videos using Anura through the endpoints we provide.
Note: Video generation can take anywhere between 4-8 mins to produce a video
Retrieve the list supported video generation models
GET /api/v1/video/models
Currently we support wan2.1; however, we are always adding new models, so stay tuned!
Request Headers
Content-Type: application/json*
Authorization: Bearer YOUR_API_KEY*
Request Sample
Response
Response Codes
200 OK: Request successful
401 Unauthorized: Invalid or missing API key
500 Internal Server Error: Server error processing request
Send out a request to create an AI generated video
POST /api/v1/video/create-job
Request Headers
Content-Type: application/json*
Authorization: Bearer YOUR_API_KEY*
Request Parameters
model*
Model used to generate the response (e.g. wan2.1). Required.
string
prompt*
The prompt input to generate your video from (max limit of 1000 characters). Required.
string
negative_prompt
An optional field to specify to the model what to exclude from the generated scene
string
Request Sample
Response
This endpoint will return an job_offer_id which is an unique identifier corresponding to the job that's running to create your video. What you'll want to do with this id is pass it into our /video/results endpoint (see below) which will provide you the output as a webp file or report that the job is still running. In the latter case, you then can continue to call the endpoint at a later time to eventually retrieve your video. As mentioned in the beginning of this section, video generation can take anywhere between 4-8 mins to complete.
Response Codes
200 OK: Request successful, stream begins
400 Bad Request: Invalid request parameters
401 Unauthorized: Invalid or missing API key
404 Not Found: Requested model not found
500 Internal Server Error: Server error processing request
Retrieve your video
GET /api/v1/video/results/:job_offer_id
job_offer_id*
The id returned to you in the video creation request i.e /api/v1/video/create-jobRequired.
string
Request Headers
Content-Type: application/json*
Authorization: Bearer YOUR_API_KEY*
Response
If the video is still in the process of being generated you will see a response that looks like the following:
Response Codes
102 Processing: Request is still processing the creation of the video
200 OK: Request successful
400 Bad Request: Invalid request parameters
401 Unauthorized: Invalid or missing API key
500 Internal Server Error: Server error processing request
However, once the video has be generated you'll be returned the video in webp format with its raw bytes which you can save to a file in the following manner:
The result of the above command will be the video.webp file being saved in the directory from which you ran it from:
The Anura API enables you to generate audio from text executed through our decentralized compute network. It's really easy to get started generating your own audio using Anura through the endpoints we provide.
Note: Audio generation can take anywhere between 40 seconds to 3 mins to complete depending on the input length
Retrieve the list supported audio generation models
GET /api/v1/audio/models
Currently we support kokoro; however, we are always adding new models, so stay tuned!
Request Headers
Content-Type: application/json*
Authorization: Bearer YOUR_API_KEY*
Request Sample
Response
Response Codes
200 OK: Request successful
401 Unauthorized: Invalid or missing API key
500 Internal Server Error: Server error processing request
Send out a request to create an AI generated audio
POST /api/v1/audio/create-job
Request Headers
Content-Type: application/json*
Authorization: Bearer YOUR_API_KEY*
Request Parameters
model*
Model used to generate the response (e.g. kokoro). Required.
string
input*
The prompt input to generate your audio from (max limit of 420 characters). Required.
string
voice*
The voice to use when generating the audio sample. Possible values are heart, puck, fenrir, and bellaRequired.
string
Voice samples
Heart
Puck
Fenrir
Bella
Request Sample
Response
This endpoint will return an job_offer_id which is an unique identifier corresponding to the job that's running to create your audio. What you'll want to do with this id is pass it into our /audio/results endpoint (see below) which will provide you the output as a wav file or report that the job is still running. In the latter case, you then can continue to call the endpoint at a later time to eventually retrieve your audio. As mentioned in the beginning of this section, audio generation can take anywhere between 40 seconds to 3 mins to complete.
Response Codes
200 OK: Request successful
400 Bad Request: Invalid request parameters
401 Unauthorized: Invalid or missing API key
404 Not Found: Requested model not found
500 Internal Server Error: Server error processing request
Retrieve your video
GET /api/v1/audio/results/:job_offer_id
job_offer_id*
The id returned to you in the audio creation request i.e /api/v1/audio/create-jobRequired.
string
Request Headers
Content-Type: application/json*
Authorization: Bearer YOUR_API_KEY*
Response
If the audio is still in the process of being generated you will see a response that looks like the following:
Response Codes
102 Processing: Request is still processing the creation of the audio
200 OK: Request successful
400 Bad Request: Invalid request parameters
401 Unauthorized: Invalid or missing API key
500 Internal Server Error: Server error processing request
However, once the audio has be generated you'll be returned the audio in wav format with its raw bytes which you can save to a file in the following manner:
The Anura API provides developers with a web search capability enabling you to add a powerful tool to your AI Agent building arsenal. LLM's are only as great as their training data and are taken to the next level when provided with additional context from the web. With web search you can power your AI Agent workflow with live web search data providing your LLM the most up to date information on the latest on goings in the world.
It's easy to get started searching the web through the Anura API using our endpoint:
POST /api/v1/websearch
Request Headers
Content-Type: application/json*
Authorization: Bearer YOUR_API_KEY*
Request Parameters
query*
The web search query you wish to execute
string
number_of_results*
The number of search results you want returned (limited to 1 to 10 inclusive)
number
Request Sample
Response Sample
The response will include the following fields:
results
The array of search results where each result object is made up of the strings: title, url and description
related_queries
An array of strings containing similar queries based on the one you supplied
count
The number of search results returned
Response Codes
200 OK: Request successful, stream begins
400 Bad Request: Invalid request parameters
401 Unauthorized: Invalid or missing API key
404 Not Found: Requested model not found
500 Internal Server Error: Server error processing request
GET /api/v1/jobs/:id - Get status and details of a specific job
You can use another terminal to check job status while the job is running.
Once your job has run, you should get output like this:
POST /api/v1/cowsay - Create a new cowsay job
Request body: {"message": "text to display"}
GET /api/v1/cowsay/:id/results - Get results of a cowsay job
Introduction to running AI models on Lilypad with the Module Marketplace
There are two ways that participants in the network can interact with Lilypad Modules: As Job Creators who run modules, or as Module Creators who create modules.
Running Jobs (participating as a 'Job Creator') is just a few simple steps:
Choose a module to run and send a job request
Try out some of the Lilypad team modules:
Run Llama 2 on the Lilypad network
Example:
Ensure you have the necessary permissions and resources to run Docker containers with GPU support.
The module version (6d4fd8c07b5f64907bd22624603c2dd54165c215) may be updated. Check for the latest version before running.
Adjust port mappings and volume mounts as needed for your specific setup.
To view the results in a local directory, navigate to the local folder provided by the job result.
How to build your own compute job for Lilypad
A Lilypad module is a Git repository that allows you to perform various tasks using predefined templates and inputs. This guide will walk you through creating a Lilypad module, including defining a JSON template, handling inputs, and following best practices.
Below are a few examples of modules you can run on Lilypad. From language models to image generators and fun utilities, the network supports a growing list of AI modules.
Start by creating a Git repository for your Lilypad module. The module's versions will be represented as Git tags. Below is the basic structure of a Lilypad Module.
Download model files
Handle all dependencies (requirements.txt)
Implement input/output through environment variables
Write outputs to /outputs directory
To use a model offline, you first need to download it and store it in a local directory. This guarantees that your code can load the model without requiring an internet connection. Here's a simple process to achieve this:
Install required libraries
Use a script to download the model (eg: python download_model.py)
Verify that the model files are in your directory
To make sure your Docker image is compatible with Lilypad, you need to define the architecture explicitly during the build process. This is particularly important if you are building the image on a system like macOS, which uses a different architecture (darwin/arm64) than Lilypad's infrastructure (linux/amd64).
The examples below are for building, tagging and pushing an image to DockerHub, but you can use any platform you prefer for hosting the image.
For Linux: docker buildx build -t <USERNAME>/<MODULE_NAME>:<MODULE_TAG> --push .
For MacOS:
Format in template:
Usage in CLI:
During development, you will need to use the Git hash to test your module. This allows you to verify that your module functions correctly and produces the expected results.
Below is a working Lilypad module run cmd for reference. (you can use this to run a Lilypad job within the Lilypad CLI):
Use the following command syntax to run your Module on Lilypad Testnet.
Here are some example Lilypad modules for reference:
Deprecated examples:
These examples can help you understand how to structure your Lilypad modules and follow best practices.
In this guide, we've covered the essential steps to create a Lilypad module, including defining a JSON template, handling inputs, and testing your module. By following these best practices, you can build reliable and reusable modules for Lilypad.
For more information and additional examples, refer to the official Lilypad documentation and the Cowsay example module.
Run a cowsay job
cowsay is a simple, text-based program originally written for Unix-like operating systems that generates ASCII pictures of a cow with a speech bubble containing a specified message.
This module was created as a "Hello World" for the Lilypad Network!
Once you've installed the CLI, run the cowsay command:
To view the results in a local directory, navigate to the local folder.
Here, you can view the stdout and stderr as well as the outputs folder for the run:
Use create-lilypad-module to create Lilypad modules
create-lilypad-module is an officially supported package that provides a simple scaffolding for building Lilypad modules. It offers a modern Docker setup with minimal configuration.
The following guide is using the "Hugging Face" template, which is more advanced.
If you are new to module building, it is highly recommended you use the "Ollama" template first.
There is currently not a guide in our docs for the "Ollama" template. We will work on adding one soon, but the directions in the README should be sufficient.
A Lightweight Stable Diffusion Module for Lilypad
Example:
Ensure you have the necessary permissions and resources to run Docker containers with GPU support.
The module version (ae17e969cadab1c53d7cabab1927bb403f02fd2a) may be updated. Check for the latest version before running.
Adjust port mappings and volume mounts as needed for your specific setup.
To view the results in a local directory, navigate to the local folder.
Create your Lilypad module
Input:
Output:
If you prefer to follow along with a video guide, you can view our live workshop below! 👇
The first thing you'll need for your module is a local model to use.
You can get started by attempting to run the download_models.py script.
Since the script hasn't been properly configured yet, it will return an error and point you to the file.
Open scripts/download_models.py and you will see some TODO comments with instructions. Let's go through them each in order. You can remove each TODO comment after completing the task.
First we have a reminder to update our requirements.txt file, which is used by the Dockerfile to install the module's dependencies. In the next line is a commented out import statement.
For now, let's look at the top 2 lines of the provided code block:
Notice that torch is also being used. Copy the transformers import statement and paste it over the existing import statement in our download_models.py file.
Now open requirements.txt:
These are 2 of the most common libraries when working with models. Similar to the import statement in the download_models.py file, they are provided by default for convenience, but commented out because although they are common, not every model will use them.
Since this model happens to use both of these libraries, we can uncomment both lines and close the file after saving.
Return to the download_models.py file, and look for the next TODO comment.
For our use case, it should look like this:
You're almost ready to download the model. All you need to do now is replace the following 2 lines after the TODO comment:
Instead of using AutoTokenizer and AutoModelForSequenceClassification, replace those with the DistilBertTokenizer and DistilBertForSequenceClassification we imported.
The script is now configured! Try running the command again.
The models directory should now appear in your project. 🎉
Now for the fun part, it's time to start using the model!
This time we'll get started by running the run_module script.
You should see an error with some instructions.
Let's tackle the run_inference.py script first. This is where your modules primary logic and functionality should live. There is a TODO comment near the top of the file.
We've already updated the requirements.txt file, so we can skip that step. Go ahead and uncomment the import statements and replace the transformers line with the DistilBertTokenizer and DistilBertForSequenceClassification.
We should refer back to the "How to Get Started With the Model" section of Distilbert's model card to figure out how to use the model.
Let's implement this into our run_inference script. Scroll down to the main() function and you'll see another TODO comment.
Same as before, uncomment and replace AutoTokenizer with DistilBertTokenizer and AutoModelForSeq2SeqLM with DistilBertForSequenceClassification. This is now functionally identical to the first 2 lines of code from Distilbert's example.
Below that, the tokenizer and model are passed into the run_job() function. Let's scroll back up and take a look at the function. This is where we'll want to implement the rest of the code from Distilbert's example. The inputs are already functionally identical, so let's adjust the output.
From the Distilbert model card, copy all of the code below the inputs variable declaration, and paste it over the output variable declaration in your modules code.
All we need to do from here is set the output to the last line we pasted.
That's everything we'll need for the modules source code!
We still need to finish step 2 that the error in the console gave us earlier. Open the run_module.py script.
Find the TODO comment and delete the code block underneath.
Before you are able to run your module, we need to build the Docker image. You can run the following command:
You should see the following response in the console:
Open the constants.py file, it should look like this:
For now, we'll be testing the module locally, so all we need to worry about is the DOCKER_REPO variable. We'll use MODULE_REPO when it's time to run the module on Lilypad Network. For help or more information, view the configuration documentation
You should be able to successfully build the Docker image now.
It's finally time to see your module in action.
Let's start by running it locally.
The CLI should ask you for an input. Enter whatever you like and hit enter. The module will analyze the sentiment of your input and output the results at outputs/result.json.
You just used a local LLM! 🎉
Before you can run the module on Lilypad Network, you'll need to push the Docker image to Docker Hub.
While the Docker image is being built and pushed, you should configure the rest of the variables in constants.py. Make sure that you push your code to a public GitHub repository.
The last thing we'll need to do is edit the Lilypad module configuration file, lilypad_module.json.tmpl. For the purposes of this module, the default configuration is mostly correct. However, the "Image" field needs to be configured.
Replace the default value with your Docker Hub username, module image, and tag.
Once your Docker image is pushed to Docker Hub and your most recent code is pushed to a public GitHub repository, you can test your module on Lilypad's DemoNet by replacing the --local flag with --demonet
You can also remove the
--demonetflag and supply yourWEB3_PRIVATE_KEYto run the module on Lilypad's IncentiveNet.
You just used an LLM on Lilypad's decentralized network! 🎉
Now anyone who has the Lilypad CLI installed can also run your module:
Configure your Lilypad module
After bootstrapping your module, additional configuration is required to run it.
.envWEB3_PRIVATE_KEY🚨 DO NOT SHARE THIS KEY 🚨
The private key for the wallet that will be used to run the job.
This is required to run the module on Lilypad Network.
A new development wallet is highly recommended to use for development. The wallet must have enough LP tokens and Arbitrum Sepolia ETH to fund the job.
config/constants.pyDOCKER_REPOThe Docker Hub repository storing the container image of the module code.
This is required to push the image to Docker Hub and run the module on Lilypad Network.
e.g. "<dockerhub_username>/<dockerhub_image>"
DOCKER_TAGThe specific tag of the DOCKER_REPO containing the module code.
Default: "latest"
MODULE_REPOThe URL for the GitHub repository storing the lilypad_module.json.tmpl file. The visibility of the repository must be public.
The lilypad_module.json.tmpl file points to a DOCKER_REPO and Lilypad runs the module from the image.
e.g. "github.com/<github_username>/<github_repo>"
TARGET_COMMITThe git branch or commit hash that contains the lilypad_module.json.tmpl file you want to run.
Use git log to easily find commit hashes.
Default: "main"
Your module will be bootstrapped with some handy scripts to help you download the model(s) for your module, build and push Docker images, and run your module locally or on Lilypad Network. Some additional configuration may be required.
In the project directory, you can run:
python -m scripts.download_modelsNo matter which model you are using, be sure to thoroughly read the documentation to learn how to properly download and use the model locally.
python -m scripts.docker_buildBuilds and optionally publishes a Docker image for the module to use.
For most use cases, this script should be sufficient and won't require any configuration or modification (aside from setting your DOCKER_REPO and DOCKER_TAG).
--push Flag
Running the script with --push passed in pushes the Docker image to Docker Hub.
--no-cache Flag
Running the script with --no-cache passed in builds the Docker image without using the cache. This flag is useful if you need a fresh build to debug caching issues, force system or dependency updates, pull the latest base image, or ensure clean builds in CI/CD pipelines.
python -m scripts.run_moduleThis script is provided for convenience to speed up development. It is equivalent to running the Lilypad module with the provided input and private key (unless running the module locally, then no private key is required). Depending on how your module works, you may need to change the default behavior of this script.
--local Flag
Running the script with --local passed in runs the Lilypad module Docker image locally instead of on Lilypad's Network.
--demonet Flag
Running the script with --demonet passed in runs the Lilypad module Docker image on Lilypad's Demonet.
lilypad_module.json.tmplThe default lilypad_module.json.tmpl file is below. Make sure to update the Docker Image to point to your Docker Hub image with the correct tag.
The default
lilypad_module.json.tmplshould work for low complexity modules. If your module requires additional resources (such as a GPU) make sure to configure the applicable fields.
Machine: Specifies the system resources.
GPUs: Specifies the minimum VRAM required.
Job: Specifies the job details.
APIVersion: Specifies the API version for the job.
Metadata: Specifies the metadata for the job.
Spec: Contains the detailed job specifications.
Deal: Sets the concurrency to 1, ensuring only one job instance runs at a time.
Docker: Configures the Docker container for the job
WorkingDirectory: Defines the working directory of the Docker image.
Entrypoint: Defines the command(s) to be executed in the container as part of its initial startup runtime.
EnvironmentVariables: This can be utilised to set env vars for the containers runtime, in the example above we use Go templating to set the INPUT variable dynamically from the CLI.
Image: Specifies the image to be used (DOCKERHUB_USERNAME/IMAGE:TAG).
Engine: Sets the container runtime (Default: "Docker").
Network: Specifies that the container does not require networking (Default: "Type": "None").
Outputs: Specifies name and path of the directory that will store module outputs.
Resources: Specify additional resources.
Timeout: Sets the maximum duration for the job. (Default: 600 [10 minutes]).
The folder structure output from using `create-lilypad-module`
After creation, your project should look like this:
For the module to run, these files must exist with exact filenames:
src/run_inference.py
The Dockerfile ENTRYPOINT.
If you change this files name or location, you must also update the ENTRYPOINT in your Dockerfile and lilypad_module.json.tmpl file to match.
config/constants.py
The configuration file that stores the DOCKER_REPO, DOCKER_TAG, MODULE_REPO, and TARGET_COMMIT.
If you change this files name or location, you must also update the import statements in scripts/docker_build.py and scripts/run_module.py.
Dockerfile
Required to build your module into a Docker image, and push the image to Docker Hub where it can be accessed by Lilypad Network.
requirements.txt
Used by the Dockerfile to install dependencies required by your module.
Technically, this file can be deleted or renamed, but this naming convention is highly recommended as an industry standard best practice.
lilypad_module.json.tmpl
The Lilypad configuration file.
You can delete or rename the other files.
You may create subdirectories inside src. For faster builds and smaller Docker images, only files inside src are copied by Docker. You need to put any files required to run your module inside src, otherwise Docker won’t copy them.
You can create more top-level directories. They will not be included in the final Docker image so you can use them for things like documentation.
If you have Git installed and your project is not part of a larger repository, then a new repository will be initialized resulting in an additional top-level .git directory.
Run the Lilypad CLI wrapper locally
Note: This is a beta tool and would mostly be expected to run locally. When implementing this tool, note that the POST request includes the user's Web3 private key. Stay tuned for a hosted API from Lilypad that will supplement this local CLI Wrapper.
Build the Lilypad binary:
In inputs, each input must be preceded by the -i flag, including tunables. For example: "inputs": "-i Prompt='an astronaut floating against a white background' -i Steps=50"
Note: This tool is for demonstration purposes and can be used to run jobs on Lilypad for free. The tooling will be improved upon in the coming weeks for greater scalability. Use the following post request with the WEB3_PRIVATE_KEY below to run jobs on Lilypad. The wallet/private key below is funded with testnet tokens only and has been setup to simplify the use of this developer tool.
The endpoint can then be tested using curl
Projects built with Lilypad
Get an API key from the .
An object specifying the format that the model must output. .
If set to true, the model response data will be streamed to the client as it is generated using .
Lilypad makes it easy to deploy and run AI models with an AI model hub called the .
Currently setup in the , the Module Marketplace makes it easy for AI model creators to distribute their work and for the community to quickly get started running a wide range of open source AI models.
Use the Lilypad or
Additional modules from the community are available in the .
A is a standard containerized (Docker) process for running compute workloads on Lilypad.
Modules can be developed by any member of the community (participating as a 'Module Creator'). If there is an AI model (or other compute job) you wish to run using the Lilypad Network resources, create a new Module and !
These instructions provide steps for running the Llama2 module on the Lilypad network using Docker and the Lilypad CLI. Find the module repo .
Before running llama2, make sure you have the on your machine and your private key environment variable is set. This is necessary for operations within the Lilypad network.
Learn more about installing the Lilypad CLI and running a Lilypad job with this .
For a more in-depth look at building modules, refer to this .
If you're new to Docker, consider exploring this on creating, building, and running a Docker image for a simple Hello World style application.
To view the full list of available modules on Lilypad, please check out !
If the job run appears to be stuck after a few minutes (sometimes it takes time for the Module to download to the RP node), cancel the job and try again. Open a ticket in with any issues that persist.
: Lilypad "Hello World" example
: Text to text
: Text to image generation
: An example module for LoRa training tasks.
: An example module for LoRa inference tasks.
: An example module related to DuckDB.
Before running cowsay, make sure you have the on your machine and your private key environment variable is set. This is necessary for operations within the Lilypad network.
: install and run create-lilypad-module
: output and explanation of create-lilypad-module files
: requirements and explanations of Lilypad module configuration
: a step-by-step guide on how to create a simple Lilypad module using create-lilypad-module
These instructions provide steps for running the SDXL Turbo Pipeline module on the Lilypad network using Docker and Lilypad CLI. Find the module repo .
Before running sdxl turbo, make sure you have the on your machine and your private key environment variable is set. This is necessary for operations within the Lilypad network.
Learn more about installing the Lilypad CLI and running a Lilypad job with this .
This guide will walk you through creating a basic sentiment analysis module using and (which will be referred to as Distilbert from now on). We will be referring back to the Hugging Face page throughout this guide, so it's best to keep it open and accessible.
To build and run a module on Lilypad Network, you'll need to have the , and on your machine, as well as and accounts.
For this guide, we'll be using which requires and uses .
A basic outline for downloading a model from is provided in scripts/download_models.py. The structure of the script and the methods for downloading a model can differ between models and libraries. It’s important to tailor the process to the specific requirements of the model you're working with.
To find the dependencies that our model requires, we can refer back to Distilbert's Hugging Face page and click on the "Use this model" dropdown, where you will see the library as an option. Click it.
You should see a handy modal explaining how to use the model with the Transformers library. For most models, you'd want to use this. However, Distilbert has a specific tokenizer and model class. Close the modal and scroll to the section of the model card. We're going to use this instead.
torch is a collection of APIs for extending ’s core library of operators.
If we take a look at the , we can use the copy button next to the name of the module to get the MODULE_IDENTIFIER. Paste that in as the value.
A basic outline for downloading a model from is provided, but the structure of the script and the methods for downloading a model can differ between models and libraries. It’s important to tailor the process to the specific requirements of the model you're working with.
Most (but not all) models that utilize machine learning use the library, which provides APIs and tools to easily download and train pretrained models.
- Vitalik Buterin
- Galaxy Digital Research
Nvidia CUDA
- Juan Benet
- Juan Benet
- Juan Benet & Molly Mackinlay
- David Aronchick
- David Aronchick
is a proof-of-concept application aiming to provide an alternative ethical solution to AI-Art generation by creating a new paradigm for AI image generation that pays royalties to the original creator and offers a new revenue stream for artists to explore.
To achieve this, Waterlily leverages the transparency & trustlessness of blockchain, with smart contracts, combined with a decentralised edge compute platform in to deliver an intuitive user experience for generative AI-Art while compensating creators for their original work with every image generation call.
Each stable diffusion model on Waterlily is trained & finetuned to a selection of artist work that has been uploaded by the artist themselves or is available in the public domain. When a user navigates to the platform, they simply enter a text prompt for the images they want to create and choose an artist style that appeals to them. The small fee paid by the user is then distributed to the artist's wallet (except the fees required by the network to run the contract calls & return the generated images) - or to a vetted creator foundation in the case of public images.
The Lilypad CLI wrapper can run locally to create an API endpoint for running jobs on the Lilypad network. This gives developers full control to build a decentralized system running jobs on Lilypad. Github repo can be found .
Build a or AI agent workflow that uses this API endpoint for running jobs! For inspiration, check out JS CLI wrapper + Gradio example. Spin up a Webui with Gradio and use the api with a frontend!
Run node src/index.js to create a local endpoint using the js wrapper with either src/run.js or src/stream.js, then send a post request containing json with your funded WEB3_PRIVATE_KEY key set, see the quick start for more on .
Llama2
An open-source language model, used for generating and understanding text.
Stable Diffusion Turbo Pipeline
A fast, high-performance version of Stable Diffusion XL for generating high-quality images with minimal latency
Cowsay
A playful CLI tool that displays text as if spoken by an ASCII-art cow
Adding Trust to the Network
Lilypad is a verifiable trustless decentralized compute network that aims to prevent cheating in the network. The network consists of clients and compute nodes. The main goal is to establish a game theoretic approach to verifiable computing, where clients can trust the results they receive from compute nodes. The approach used in Lilypad is pure verification by replication, without relying on cryptographic tools like snarks or trusted execution environments.
In the context of Lilypad, it's important to understand the difference between global consensus and local consensus. Global consensus, as seen in blockchains, ensures that every node knows the result of a computation is correct due to massive replication across many nodes. However, in Lilypad's two-sided marketplace, only the client needs to be convinced that the result is correct. Local consensus is sufficient for the client, while other nodes like verifying nodes, requesters, and market makers may also need to be convinced.
Lilypad assumes non-malicious but utility-maximizing agents, including both clients and compute nodes. Utility-maximizing means that nodes will do anything to maximize their return, including returning false results if necessary. However, they are not malicious and won't purposely send back false results. The adversary model assumes that all nodes are utility-maximizing and does not consider malicious behavior.
Lilypad aims to achieve the best outcome, where nodes never have any incentive to cheat. If that's not possible, the goal is to minimize the incentive to cheat as a function of the protocol's parameters. Another option is to achieve the first two outcomes under simplifying assumptions, such as having a fraction of honest nodes in every mediation protocol. Lilypad also considers the possibility of not being able to achieve any of these outcomes and aims to understand the limitations.
Lilypad takes an adversary-first approach by designing the most powerful adversary possible and then optimizing against it. The team uses multi-agent reinforcement learning to train agents to act as utility-maximizing agents on behalf of clients and compute nodes. Reinforcement learning has shown impressive results in various domains, and Lilypad aims to leverage its capabilities to train agents that can maximize their utility in the network.
Lilypad plans to test various anti-cheating mechanisms once the reinforcement learning agents are implemented. These mechanisms include:
Consortium of mediators: A group of trusted mediators can check the results, reducing the likelihood of cheating.
Prediction markets and staking: Nodes can stake behind other nodes and lose their collateral if the node they stake behind is found to have cheated.
Taxes and jackpots: A tax is imposed on every job, and the taxes go into a jackpot that is awarded to nodes that find other nodes to have cheated.
Reputation: Nodes can build up a reputation based on the number of jobs they've done and the honesty of their results.
Sorting inputs and outputs: Storing inputs and outputs for longer periods allows for easier verification but increases collateralization requirements.
Frequency of checking results: Autonomous agents can decide when to check results, balancing the need for verification with the cost of collateralization.
In the future, Lilypad aims to take into account the preferences of nodes, such as computational requirements, time requirements, and scheduling requirements. The long-term goal is to have compute nodes and clients negotiate with each other automatically over these aspects of a job. Lilypad acknowledges that game-theoretic verifiable computing is a less studied form of verifiable computing compared to zero-knowledge proofs and trusted execution environments. However, the team is committed to exploring this approach and conducting rigorous research to find effective solutions.
Lilypad's game-theoretic approach to verifiable computing aims to prevent cheating in a decentralized compute network. By using reinforcement learning and testing various anti-cheating mechanisms, Lilypad strives to create a trustless environment where clients can have confidence in the results they receive from compute nodes. The team is actively working on implementing reinforcement learning agents and conducting simulations to evaluate the effectiveness of different strategies.
3D Mapping on Lilypad
This is a work-in-progress proof of concept utilizing Extra Labs and Lilypad to create a Lilypad module for running multiple geospatial jobs.
The goal is to provide the end-to-end adapter for users to operate on the point cloud data to generate reconstructed maps.
Extra Labs is revolutionizing the way maps are built by leveraging collaborative acquisition and deep learning to produce affordable, detailed 3D models of cities and territories.
To achieve this proof of concept, the Lilypad module will integrate with Extra Labs' platform, allowing users to submit their geospatial data, whether collected via drones, aircraft, delivery vehicles, or smartphones. This data is processed through advanced algorithms to create detailed 3D reconstructions.
The decentralized nature of the platform ensures that data providers are compensated fairly for their contributions through a blockchain-based reward system. This approach not only democratizes access to 3D mapping technologies but also ensures continuous and up-to-date data acquisition, enhancing the accuracy and detail of the generated maps.
Users should be able to easily interact with the platform, upload their data, and receive high-quality 3D models in return. This process is designed to make advanced geospatial mapping accessible to a wide range of users, from urban planners and architects to developers and hobbyists.
An agent-based system that monitors HubSpot for recently modified contacts, fetches enriched data from LinkedIn, and updates HubSpot automatically.
This workflow serves as a proof of concept (PoC) illustrating the capabilities of agents to interface with platform APIs, specifically those of HubSpot and RapidAPI.
Python 3.8+
A HubSpot account with a Private App token
A RapidAPI account with a LinkedIn Data API key
An OpenAI-compatible API
Choose the model you would like to use
Clone the Repository
Install Dependencies
Note: If requirements.txt is missing, install:
Configure Environment Variables
Create a .env file:
Add your keys:
Start the Daemon
Checks HubSpot every minute for contacts modified in the last 7 days (or since last_processed.json).
Fetches LinkedIn data and updates HubSpot.
Runs for 5 minutes (adjustable in main.py).
Monitor Logs
Look for "Found X contacts", "HubSpot update: ...", "Attempting to save last_processed timestamp".
If no contacts are found, last_processed.json won’t save—edit a HubSpot contact to test.
main.py: Entry point and daemon logic.
/tools/hubspot_tools.py: HubSpot API utilities (fetch/update contacts).
/tools/linkedin_tools.py: LinkedIn API utility (fetch profile data).
/agents/hubspot_agent.py: Agent for updating HubSpot.
/agents/linkedin_agent.py: Agent for fetching LinkedIn data.
A retrieval-augmented generation (RAG) agent that retrieves relevant context and generates AI-powered responses using the Lilypad Network.
The Lilypad RAG Support Agent is a Retrieval-Augmented Generation (RAG) AI assistant that retrieves relevant information and generates AI-powered responses using the Lilypad Network. It enables automated support and troubleshooting by leveraging vector search and AI-based text generation.
Context-Aware Responses – Uses all-MiniLM-L6-v2 embeddings to retrieve relevant information.
AI-Powered Answer Generation – Sends retrieved context and user query to the Lilypad API, which processes it using Llama3 8B.
Customizable Knowledge Base – Modify the agent’s context source (issues.md) to adapt it for different use cases.
Embedding Queries with all-MiniLM
Converts user queries and stored knowledge into dense vector embeddings for similarity-based retrieval
Retrieving Relevant Context
Searches a pre-indexed database to find the most relevant information.
Generating Responses with Llama3 using the Lilypad API
Sends retrieved context and user prompt to the Lilypad API, where Llama3 8B generates a structured response
The Lilypad RAG Support Agent can be adapted to different projects by modifying its retrieval source.
issues.md)By default, the agent retrieves information from issues.md, a markdown file containing troubleshooting steps.
To customize:
Open issues.md in the repository.
Replace or expand the content with relevant support information for your project.
Format the content clearly to improve retrieval accuracy.
Restart the agent to index the updated knowledge base.
For more advanced use cases, the agent can be extended to support multiple files or external knowledge sources.
Export your Lilypad API Token as an environment variable:
To make it persistent, add it to ~/.bashrc or ~/.zshrc.
Ensure Python 3 is installed, then run:
After setting up the API key and dependencies, the agent is ready to process queries using Lilypad’s AI-powered retrieval system.
Execute the agent from the project's root directory:
AI tooling using Lilypad
AI tooling provides the essential infrastructure and frameworks that support the development, deployment and monitoring of AI-powered applications. This section highlights tools that help streamline workflows, enhance visibility and enable customization when building with AI on Lilypad, with examples and guides for integrating them into your projects.
A research agent template implemented as an AI oncologist. Read papers, isolate important information, and produce actionable reports.
The AI Oncologist serves as a Research agent template that can be used to conduct a wide range of research. The Agent is an intelligent system for analyzing research papers using a multi-agent approach.
The system consists of three main agents:
Searches through PDF documents in the documents directory
Uses embeddings and cosine similarity for initial filtering
Verifies relevance using LLM-based analysis
Returns a list of most relevant paper filenames
Extracts text from identified papers
Splits content into manageable chunks
Scores paragraph relevance using LLM
Returns top-scoring paragraphs with relevance scores
Analyzes provided text passages
Generates focused answers to specific queries
Uses contextual understanding to provide accurate responses
Python 3.8+
OpenAI API key or compatible API (e.g., DeepSeek)
PDF files
Clone the repository:
Install required packages:
Create a .env file in the project root with the following variables:
BASE_URL: API endpoint for OpenAI Compatible LLM service (default: "https://api.deepseek.com")
MODEL: One OpenAI Compatible model to use (default: "deepseek-chat")
ANURA_BASE_URL: API endpoint for the Anura LLM service (default: "https://anura-testnet.lilypad.tech")
ANURA_MODEL: The Anura model to use (default: "phi4:14b")
MAX_RESULTS: Maximum number of papers to return (default: 3)
SIMILARITY_THRESHOLD: Minimum similarity score for document selection (default: 0.3)
Place your PDF research papers in the documents/ directory.
Run the main script:
AI agents on Lilypad
AI agents can autonomously process information, make decisions and interact with users or systems. This section explores different use cases for AI agents on Lilypad, providing examples and step-by-step guides on how to build and run them using the network.
Tutorial: Using Lilypad x AnythingLLM for Desktop LLM Client
A basic understanding of the Lilypad API and Anything LLM
Access to an Anura Dashboard account
Start AnythingLLM App.
Navigate to the Settings page by clicking wrench icon at bottom left
Click on LLM under AI Providers
Select Generic OpenAI under LLM Provider
Click Get Started
Create a new API key by clicking the “Create API Key” button.
Copy the generated API key and paste it into the APIIey input field on the LLM Screen.
Paste in "llama3.1:8b"
A developer platform for using the Lilypad network to train and run AI models
The ML workbench provides an interface to:
Run models that are currently available on the Lilypad GPU network
Leverage Jupyter notebooks with Lilypad
Create multi-module flows
Fine tune models
Create an AI agent
Run existing Lilypad modules with a simple text or image prompt. The workbench will output an IPFS CID with the result.
Implement the Lilypad module allowlist and cache the models locally before they are needed! Pin the files to a local IPFS node packaged with the workbench platform.
Import a dataset and use the the "Data" tool to finetune a model for a specific end user. Create a Lilypad module and save the module on IPFS. More on this soon!
The ML workbench provides a simple interface for creating and training AI agents. The Lilypad team is currently testing a variety of models and fine tuning techniques to determine the optimal path forward for this feature. More on this soon!
Tutorial: Using Lilypad x Rivet for Generating Narratives with Anura
A basic understanding of the Lilypad API and Rivet
Access to an Anura Dashboard account
Basic knowledge of workflow configuration
Start Rivet App.
Navigate to the Settings page. App>Settings
Click on the “Open AI” tab.
Click Get Started
Create a new API key by clicking the “Create API Key” button.
Copy the generated API key and paste it into the Open AI API key input field in Rivet.
Built with Lilypad
This section features hands-on tools and resources designed to support developers building with Lilypad. From code editor extensions to social app integrations, it highlights ways to prototype, automate and ship AI jobs more efficiently. Explore how these utilities fit into real workflows and help extend the power of the network.
An integrated development environment for running AI workloads on the Lilypad Network directly within Visual Studio Code.
The Lilypad VS Code Extension allows developers to interact with the Lilypad Anura API within Visual Studio Code. By highlighting code blocks, users can query the available LLMs on Lilypad for explanations, improvements, or suggestions, receiving responses in a formatted webview panel.
AI-Powered Code Assistance – Select any code in your editor, choose an AI model, and ask Lilypad questions about it
Formatted Responses – View AI-generated insights in a structured webview panel
Secure API Configuration – Store your Lilypad API token securely
Context Menu Integration – Quickly access the extension’s features via the right-click menu
From the Repository
Clone the repo: git clone git@github.com:PBillingsby/lilypad-vscode-extension.git
Install the extension in VS Code: code --install-extension lilypad-vscode-extension.vsix
Add Anura API key to .env: LILYPAD_API_TOKEN=<ANURA_API_KEY>
If you make changes to the extension and need to recompile it:
Recompile the code: npm run compile
Repackage the extension: vsce package
Reinstall the updated .vsix file: code --install-extension lilypad-vscode-extension.vsix
Select a block of code in your editor.
Right-click and select "Ask Lilypad about this code", or:
Open the Command Palette (Ctrl+Shift+P) and select "Ask Lilypad about this code".
Choose an AI model to process the query.
Enter your question related to the selected code.
Wait for Lilypad AI to process your query.
View the AI’s response in the webview panel that opens.
An AI-powered chatbot that generates responses using the Lilypad API and Llama3.
The Lilypad Llama3 Chatbot is a conversational agent designed to deliver engaging, real-time AI-powered interactions. Leveraging the Llama3 8B model through the Lilypad API, this chatbot provides context-aware and dynamic responses, making it useful for applications such as automated support, interactive Q&A and virtual assistance.
AI-Powered Conversations – Utilizes Llama3 8B via the Lilypad Network to generate intelligent responses.
Lilypad API Connectivity – Simple integration with Lilypad, allowing flexible use of all available models on the Lilypad API.
This chatbot can be extended with:
Memory & Context Awareness – Store conversation history for more personalized interactions.
External APIs – Integrate with knowledge bases, search engines, or database lookups.
Multi-Model AI – Swap Llama3 with other AI models as needed.
User Input – The chatbot interface captures the user's message.
API Request – The message is sent to the Lilypad API, which runs the job on the Llama3 model.
Response Generation – Llama3 processes the input, considers context and generates a natural language response.
Response Display – The response is rendered in the chatbot interface.
Clone the repository:
Install dependencies: npm install
Add the Lilypad API key to .env: LILYPAD_API_TOKEN=<ANURA_API_KEY>
Start the development server: npm run dev
Access the chatbot at: http://localhost:3000 and enter prompt to start a conversation.
A guide for running Lilypad jobs in a Farcaster frame
In this guide we will be building a Farcaster frame for local development that runs a Lilysay prompt on the Lilypad Network.
Users of the frame can input a prompt and a generate an image. The generated image will appear in the box above the input and will allow a user to view the ASCII art as an image.
For this frame, create a NextJS 14 app:
Open the project in your code editor.
.env.localAdd the following into your .env.local file. The private key is used to run the CLI jobs. Make sure that your .env.local file is added to the .gitignore file as your private key should not be exposed or pushed.
WEB3_PRIVATE_KEY=YOUR_PRIVATE_KEY
NEXT_PUBLIC_BASE_URL=http://localhost:3000
Framegear is a simple tool provided by the @coinbase/onchainkit package that allows you to run and test your frames locally without publishing the frame.
In a separate terminal, clone down the onchainkit repo and run Framegear:
We will need to set up the metadata for our Next.js application that includes Farcaster frame information. It will configure the elements and a URL for frame posting, while also specifying Open Graph metadata for improved social sharing.
In app/page.tsx, add the following before the Home function declaration:
The UI elements for this frame are all rendered in the app/api/route.ts file, which acts as a request handler for different routes or endpoints within the web application. It defines the logic for handling user input, generating responses, and managing application state. The main functions include processing user prompts, handling status checks, and generating images asynchronously.
Here’s how the routes are structured for this frame:
/api/frame?action=input: This route displays the initial user interface, which includes a text input field for the prompt and a button to submit the form. The user interface also updates dynamically based on the processing status, such as showing a placeholder image or the final generated image.
/api/frame?action=submit: This route processes the user input. When a prompt is submitted, the server initiates the image generation process asynchronously. While the image is being generated, the user sees a loading state, and they can check the progress.
/api/frame?action=check: This route checks the status of the image generation process. It updates the frame with either a completed image, an error message if the generation fails, or the processing state if the image is still being generated.
We also include a fallback just in case an error occurs during the processing of job.
/api/frame?action=save: Though not explicitly included, this could be an additional route for handling the logic of saving the generated image to a location for future access.
generateImageThe generateImage function handles the user input and generates the final image for display by utilizing the functions in app/services/cli.ts. It ensures that the image is generated asynchronously and the result is available for display in the frame, or handled properly in case of any errors during the generation process.
Throughout the interaction process, various images are used. These images serve as visual cues during each step of the frame, such as when the user is prompted for input, while the image is being processed and once the final result is ready or if an error occurs.
route.tsCreate the api/frame directories in your project. Inside of the app/api/frame directory, create a file named route.ts and add the following code:
cli.tsThis is where the execution of the Lilysay job happens. It will run the job, wait for it to finish and then create an image from the return value.
Inside of the app directory, create a new directory named services and inside of that create a file named cli.ts. The functions inside this file will allow us to send a prompt to the Lilypad Network using the user prompt and a predefined command that runs asynchronously in the terminal. Once the command is executed, Lilypad processes the input through its Lilysay module and outputs the results in the form of an ASCII image, which is then converted into a displayable image using an SVG-to-PNG transformation.
Here are the 3 functions inside this file:
createImageBufferFromAscii: Converts ASCII text into an SVG image and then uses the sharp library to convert the SVG into a PNG image buffer. This allows the display or saving of an image representation of the ASCII text.
runCliCommand: Executes a Lilypad CLI command to process the user's input text, captures the command's output, and converts it into an image buffer. It handles the entire process of running the command, capturing the output, and managing errors.
extractStdoutFilePath: Parses the CLI command's stdout to extract the file path where the Lilypad CLI has saved the output. It uses a regex pattern to identify the path in the command's output.
The following code snippet demonstrates the process:
Sending the Request: The user's input text is passed directly into the Lilypad CLI command using a shell process. The input text is embedded within the command's arguments and executed asynchronously in the terminal.
Handling the Response: After the CLI command completes, the output is captured and processed. The response includes the file path to the generated ASCII image, which is then read from the file system and converted into a PNG image for further use.
Error Handling: If an error occurs during the execution of the CLI command or file processing, it is logged to the console, and the process is terminated with appropriate error messaging.
The Framegear server is running. Next, run your local server in your frame project. We will need to make sure it is running on port 3000:
You should now see your frame and be able to interact with it. Enter a prompt to display in the Lilysay ACSII image!
As the job is processed, a “Check Status” button will be displayed. Clicking this will check if the job has been completed. Once a job is completed, the results will be displayed in the frame. If there is an issue along the way, an error message will be displayed and the user will be prompted to try again.
Running Lilysay jobs is just one of the ways you can utilize Lilypad in a frame, but you can run jobs with any available modules on the Lilypad Network. Some of these include:
Stable Diffusion image
Stable Diffusion video
Ollama
Minimum Hardware Requirements
Processor: Quad-core x64 Processor or better
RAM: 32GB (see additional details below)
Internet: Internet: 250Mbps download, 100Mbps upload (minimum)
GPU: NVIDIA GPU with a minimum of 8GB VRAM (see additional details below)
Storage: SSD with at least 500gb of free space
Base Requirement: The simplest model on Lilypad requires a GPU with at least 8GB of VRAM. This is the minimum required to participate in computational tasks on the Lilypad network.
The capability of your GPU to manage multiple or more complex Lilypad jobs is enhanced by the amount of VRAM available:
Standard Models (SDXL, Ollama): Require at least 8GB of VRAM.
Advanced Models: Require 14GB+ of VRAM.
GPUs with 8GB of VRAM are limited to running models like SDXL, which fit within this specification. Larger GPUs with higher VRAM are required for more demanding models like SDV, which needs at least 14GB of VRAM.
Lilypad uses the Resource Provider's GPU to load models, initially requiring the temporary storage of the data in the system's RAM. In a production environment with RP Nodes, it is important to have enough RAM to support the model and the underlying system's operational processes.
Base Requirement: A minimum of 16GB of RAM is required, with at least 8GB dedicated to the model and another 8GB allocated for system operations and Lilypad processes.
Wallets for each GPU: You need a separate account for each GPU you want to set up on the network. The wallet you use for your account must have both ETH (to run smart contracts on Ethereum) and Lilypad (LP) tokens in order to receive funds for jobs) on the network.
Larger Models: Jobs involving more substantial models will demand additional RAM. It's important to note that adequate VRAM alone is insufficient; your system must also have enough RAM to load the model into the GPU successfully. Without sufficient system RAM, the model cannot be loaded into the GPU, regardless of available VRAM.
A set of n8n workflows using Lilypad AI Models
At Lilypad, we’ve integrated n8n with the Lilypad API to automate dynamic workflows, combining human input, AI generation and multi-platform actions.
Using Lilypad’s OpenAI-compatible endpoints inside n8n lets you:
🧠 Source and enrich content: Pull from databases like Notion, Airtable, or Google Sheets and enhance it with AI.
🎨 Generate custom outputs: Create AI-written text, summaries, or even modify images dynamically.
🔄 Automate publishing: Push results to platforms like Twitter, Discord, Slack, or your own apps.
📈 Track and update: Monitor workflow status and feed results back into your systems.
Everything runs automatically inside n8n, with Lilypad API endpoints providing powerful, cost-free AI capabilities at the heart of it.
Imagine building workflows that review customer feedback with AI, generate personalized emails, summarize research papers, trigger alerts from on-chain events, or even create full content campaigns, all without writing a single script. With n8n and Lilypad, AI-driven automation is at your fingertips.
Coming soon:
Instructions to update the Docker Lilypad Resource Provider (RP)
To update a Lilypad RP, remove any previous versions of Lilypad on the instance and then follow the instructions to setup a Docker RP.
This only applies if you had a Linux version installed
If they are running, stop them with:
Remove the containers:
Delete old Docker images that are duplicates for Lilypad (Bacalhau, Lilypad)
curl to download the docker-compose.yml file from the Lilypad GitHub repository.OR start with your own RPC URL
Use the following command to check the status of the resource provider and bacalhau.
Use the following command to view the containers running after starting Docker Compose.
A healthy, updated node should have all containers started, a preflight check, and be adding a resource offer.
Validation of GPU Computation in Decentralized, Trustless Networks
Verifying computational processes in decentralized networks poses a fundamental challenge, particularly for Graphics Processing Unit (GPU) computations. The Lilypad Research team conducted an investigation revealing significant limitations in existing approaches: exact recomputation fails due to computational non-determinism across GPU nodes, Trusted Execution Environments (TEEs) require specialized hardware, and Fully Homomorphic Encryption (FHE) faces prohibitive computational costs.
To address these challenges, this report explores three verification methodologies adapted from adjacent technical domains: model fingerprinting techniques, semantic similarity analysis, and GPU profiling. Through systematic exploration of these approaches, we develop novel probabilistic verification frameworks, including a binary reference model with trusted node verification and a ternary consensus framework that eliminates trust requirements.
These methodologies establish a foundation for ensuring computational integrity across untrusted networks while addressing the inherent challenges of non-deterministic execution in GPU-accelerated workloads.
Common FAQs when running a Lilypad node
Without a discussion opened, our team will not be able to support the problem.
Description (including Lilypad version running on your node)
Hardware Info (including Linux/Windows version)
Related blockchain/ETH addresses of transaction hashes
Output Logs - sudo systemctl status lilypad-resource-provider
Related links/urls
Screenshots
Updating and restarting the Lilypad services regularly (daily) is encouraged throughout IncentiveNet.
To check if the RP is running use the following command:
This will give a live output from the Lilypad node. The logs will show the node running and accepting jobs on the network.
Run the following command to get info from Bacalhau
Resource Providers can run on the Lilypad Network without a GPU, however only hardware with a GPU is currently rewarded with Lilybit_ rewards.
To verify your Docker runtime configuration: sudo docker info | grep Runtimes
You should see the NVIDIA runtime listed. If you only see: Runtimes: io.containerd.runc.v2 runc you will need to configure the NVIDIA runtime.
If the NVIDIA runtime is not showing up or you're experiencing issues, try the following:
1. Configure the NVIDIA Container Toolkit runtime: sudo nvidia-ctk runtime configure --runtime=docker --set-as-default
2. Restart the Docker service: sudo systemctl restart docker
For a comprehensive overview of your Docker setup, use: docker info. This command provides detailed information about your Docker daemon configuration.
Typically this occurs when an old version of Lilypad is still running on the instance.
Ensure the Bacalhau and Lilypad systemd services are stopped and removed.
Disable the systemd services so they start on boot:
Delete the service files from the systemd directory.
**Note: Be extremely careful when using sudo to remove files.
Reload the systemd daemon to apply the changes.
Lilypad RPs currently only support Linux installs. Running a RP on Windows is currently experimental.
No, this would be considered detrimental to the Lilypad network and cannot be rewarded at this time.
In order to run jobs successfully, Lilypad RPs must ensure all resources on the server are available. If a single GPU is running many RPs and splitting the server resources between them, this causes each RP to not have enough resources to run a Lilypad job.
Here's a quick guide on setting up your own RPC for a Lilypad node.
Indicates that the CUDA version of the RP is incorrect. Install the CUDA version which suitable for the gpu type and compile Lilypad by themselves.
Resource providers are expected to have the latest Lilypad version installed on their systems. The installation instructions can be found here:
Currently, it's not possible. However, it's a very good feature request and the team is evaluating!
This is more than likely due to you trying to export your mnemonic seed phrase instead of the private key. A private key typically appears like this: 4c0883a69102937d6231471b5dbb6204fe512961708279df95b4a2200eb6b5e7 and consists of 64 hexadecimal characters.
Become an early RP on the Lilypad Network and power the future of Open Access AI
Read more about contributing hardware and earning rewards in Lilypads ongoing RP-Beta Program.
To find out more, please visit the .
Automate customer relationship management, internal company management, etc. with the ! Scale inference using the Lilypad network deploying any of the models supported by the .
Github repo .
A guide to launch the HubSpot-LinkedIn agent locally and run inference on the Lilypad Network with the .
Steps: Log in > Settings > Integrations > Private Apps > Create private app > Name it > Scopes: crm.objects.contacts (read/write contacts) > Create app > Click app name > Auth tab > View access token > Copy token
Steps: Create an account > Subscribe to > Go to > Select "default-application" > Copy "Application Key"
(Sign up and generate an API key)'
A guide to launch the RAG support agent locally and run inference on the Lilypad Network with the .
Sign up at and generate an API key.
A guide to launch the Research agent locally and run inference on the Lilypad Network with the .
OPENAI_API_KEY: Your OpenAI Compatible API key (for example: )
ANURA_API_KEY: Your Anura API key (get it here: )
Download and Install AnythingLLM from
In the BaseURL input field, paste the URL:
Log in to your Anura account at
The provides ML researchers, businesses, and more a simple interface for leveraging the Lilypad network.
Add a new model to run on the network as a
In order to run an AI model on the Lilypad network, a docker image must be created for the program using the .
The ML workbench product is currently in beta testing and is not yet provided as a hosted solution. Reach out in the Lilypad for more info!
Download and Install Rivet from
In the Open AI endpoint input field, paste the URL:
Log in to your Anura account at
Need some inspiration? Visit the page!
Get .
Get .
This is an example for running a local version as inspiration for developers. The Lilypad CLI will need to be wrapped and the project hosted as an API to run in production and be used in a .
Next, install the required dependencies. is used in this project:
We will need to fund a wallet with Lilypad and Arbitrum Sepolia testnet tokens. Follow the first 2 sections labelled "Setting up MetaMask" and "Funding your wallet" from our docs.
Your WEB3_PRIVATE_KEY can be retrieved from the MetaMask account details menu. For more info, check out the on how to get a your private key. Please do not push your private key to GitHub.
Run through this guide to . Select the CLI User script.
Navigate to and keep that window open for when we start to write the frame.
To obtain the images used in this guide, save them from .
Navigate to the Framegear host and you will see an input labeled "Enter your frame URL". Add http://localhost:3000 to that and click "Fetch".
This page overviews the hardware requirements to operate a Lilypad Network node. It's important to note that these requirements continuously evolve as the network grows. If you have questions or suggestions, please join our or open a pull request on the .
Each model operating on Lilypad has specific VRAM (Video Random Access Memory) requirements directly related to the model's complexity and computational demands. For running a Lilypad Resource Provider (RP) with multiple GPUs, a guide using Proxmox can be found .
A node with a GPU containing 8GB of VRAM can execute Lilypad module jobs, which require a minimum of 8GB of VRAM.
Larger capacity GPUs are needed for heavier compute models like , which require at least 14GB of VRAM.
If the Lilypad install for Linux was previously used on the RP, first . Then follow the .
If the RP previously used a Docker Lilypad version,, then follow the .
If a wallet has been used to run CLI jobs on Lilypad, this wallet cannot be used for a RP). See the Docs for
The Lilypad Research team published "" diving into the complexities of validating successful compute jobs on GPUs. The investigation then explores verification methods and solutions that Lilypad can implement across the network.
(under maintenance)
For complex issues, bug reports, or feature requests, open a discussion in the Lilypad-Tech Github organization discussion .
Navigate to the discussion , select "New Discussion", choose "rp-issues", and fill out the template.
For quick questions or minor issues, use the Lilypad Discord channel and provide the following info.
Ensure the RP (node) is running the latest for your preferred environment
Does the RP have Lilypad Tokens (LP) and Arbitrum ETH?
The Discord channel is the primary location for Resource Provider announcements. Announcements in this channel are also posted on the Lilypad page.
Lilypad supports Resource Providers using the Docker find instructions .
If this doesn't solve the problem, with our team.
Recommendation guide using Proxmox found . More on this coming soon!
The CUDA version of the RP does not match the GPU driver. Please refer to this to repair installation.
When do Lilybit_ rewards earned or rewards slashed appear in the Leaderboard? **The leaderboard is currenlty under maintanence, find Lilybits earned by here.
To stay up to date with the latest releases, check the in the Lilypad Discord or visit the . Along the top menu, click the "Watch" dropdown and you will see a section named "Custom". Selecting "Releases" will allow you to get notified of any new releases!
Check out the .
View the .
a wallet with LP and ETH
No ETH or LP in your wallet? ()
Join our for more help!
Become a Resource Provider:
Marketing Workflows
Life workflows
The Distributed Compute Problem
The setup is a trustless, permissionless, two-sided marketplace for compute, where clients can purchase compute services from compute nodes. Trustless means that by default, the protocol does not assume that any particular node behaves in a trustworthy manner and that each node should be considered as rationally self-interested (note that this excludes intentionally malicious behavior). Permissionless means that any node can join or leave the network at will.
Matches for compute jobs are made off-chain, with the resulting deals and results recorded on-chain. Both clients and compute nodes need to agree to matches before they become deals, and make deposits to the protocol to enable rewards and punishments. Results are verified using verification-via-replication, and clients can check the results of any job after it has been completed, but before it needs to pay. It does so by calling upon a mediation protocol. The mediation protocol is the ultimate source of truth, and the outcome of the mediation protocol determines how payouts to nodes are made.
The issue of preventing fake results in the presence of a Trusted Third Party (TTP) as a mediator is effectively a solved problem (for example, see the section on prior verification-via-replication protocols, though there is much more literature on this topic). Given the assumption that the mediation protocol is the source of truth, we can treat the mediation protocol as a TTP. Since the fake results problem is basically already solved in this case, the cheating problem reduces down to solving the collusion problem within the mediation protocol. (Note, however, that we will address both cheating and collusion; the framework described here exists to conceptually simplify the problem.)
This is a typical scenario of a Byzantine environment, and we can use well-established approaches to Byzantine Fault Tolerance when invoking mediation. However, most BFT algorithms and cryptographic methods rely on assumptions regarding some fraction of honest nodes. The problem is that rational, utility-maximizing agents may still collude, even in a mediation consortium, in order to converge on incorrect results. On the one hand, we could assume that some fraction of nodes are honest, as is often done. On the other hand, can we do better?
The task is to find the mechanisms that incentivizes all nodes to behave honestly.
All agents in the protocol are utility-maximizing. This will be elucidated in a subsequent section. Most of the literature has focused on the case where compute nodes are dishonest. However, the client can also behave dishonestly in some manner that maximizes their utility. For example, if the client has some level of control over the choice of mediator, and dishonest nodes have their collateral slashed, the client could collude with the mediator in order to deem a correct result incorrect and get a cut of the honest compute node's collateral.
"Good" solutions can take a number of forms:
Nodes never have an incentive to be dishonest.
Nodes have an incentive to be dishonest that goes to zero as a function of the parameters of the protocol.
(1) or (2), but under some simplifying assumptions, such as there being some fraction of honest nodes within every mediation protocol.
A good solution would achieve any of these goals. Alternatively, another valuable outcome of this research would be to discover under what assumptions these goals can or cannot be met, or if the goals are even possible to achieve at all.
There are a number of ways that these goals may be achieved. The approach will be to construct a digital twin of the protocol and test a number of different mechanisms in simulation. These mechanisms include variations on mediation protocols, collateralization, staking, taxes and jackpots, and others; see the Mechanisms to Explore section for details.
The following is a list of mechanisms that we are currently considering exploring in order to mitigate attacks. Note that some of these mechanisms clearly would not be able to deter cheating and collusion alone. However, in combination with other mechanisms, they may achieve the goals. In this sense, they should be thought of as modules, optionally composable with each other.
Clients call upon a mediation protocol in order to verify the results of a node. There are several variations of the structure of the mediation protocol; the following parameters can be varied:
The number of nodes in the mediation protocol.
If more than two nodes in the mediation consortium, the consensus threshold that determines which result is the one considered to be correct.
How the nodes are chosen.
For example, we may want as a baseline the same constraint as in Modicum - that only mediators that both the client and the compute node mutually trust can be used for mediation.
Even with this baseline, there is still a question of how to choose the node(s) - it can be random, be determined by an auction, or any other method.
Recursive mediation - that is, if there is no consensus in the consortium, do another mediation.
There is a large space of possibilities regarding how to execute this.
There needs to be a limit to the number of nodes this recursive process can use. For example, the set of potential nodes can be the same as the set of mutually trusted mediators, as described above.
Other methods discussed here, such as taxes and jackpots, as well as staking and prediction markets, can be incorporated into the mediation protocol.
There are a number of different types of collateral that need to be included in the protocol.
The client needs to deposit collateral so that the compute node knows that it can be paid. For computations where the cost is known up front, this is simple. However, it becomes complicated for arbitrary compute; the client might not have put up enough collateral initially, so there may have to be a back-and-forth between client and compute node where the latter halts the computation until the former deposits more collateral or simply cancels the computation and pays for the partially computed result. If the client does not pay, then the compute node can invoke a mediation process.
The compute node needs to deposit several types of collateral.
Collateral in case they timeout.
This is put up as part of the deal agreement - that is, when the deal is posted on-chain.
Collateral in case they cheat.
The way that the compute node will convey the amount of collateral they will deposit to indicate that they will not cheat is via a collateral multiplier. The compute node commits to a multiple of whatever they will charge the client ahead of time as part of the deal agreement. The actual collateral is put up after the result is computed and sent to the client. This simplifies immensely the task of determining how much collateral to deposit for arbitrary computations.
Collateral in case they don't do the computation at the rate they said they would. This is closely related to timeout collateral.
Ideally, this is a way of enforcing deadlines on jobs.
It is not necessary to make this collateral slashing binary - for example, a late result can be still be rewarded.
It is enforceable if, for example, the compute node says that they will do X WASM instructions/time. However, technical limitations may make this unrealistic, and it needs to be tested.
One possible way to overcome collusion is to require super high collateral for some particular nodes against each other that that even discount factors very favorable to collusion would not incentivize collusion, even when accounting for repeated games.
While this is not a part of anti-cheating mechanisms, collateral pooling could lower capital requirements for collateralization. High capital requirements are a second-order concern, but will become a higher priority once robust anti-cheating mechanisms are implemented.
Taking inspiration from the taxes and jackpots scheme used in Truebit, deals can be taxed, with those taxes going to a jackpot that is then used to reward nodes via some distribution protocol determined by the mediation process. For this, we want to be able to take any fraction of the jackpot(s) and distribute it arbitrarily to arbitrary nodes (perhaps even those not involved in the mediation process).
This is a particularly interesting approach because the taxation + jackpots mechanism inherently create a global game that impacts local outcomes. While it may lead to potential collusion attacks, the tool alone is very useful, especially in conjunction with some other the other methods mentioned here. Modeling it in simulation would also provide the opportunity to test some of the hypotheses in the Truebit paper.
This method may also be useful in creating a robust platform where some clients do not care to check their results. That is, if some clients do not check results in general, it may be difficult to assert that the network is secure. Taxes and jackpots may be a way to address this.
Prediction markets have been well-studied in a variety of different fields. More recently, a type of prediction market called a replication market has been explored in the context of replicability in science. With this inspiration, it may be possible that allowing nodes to make bets regarding the replicability of the computations of nodes may be useful in mitigating cheating. For example, nodes with a low prediction for replicability may act as a signal for that node's reputation and encourage it to behave honestly.
It is possible to overlay this mechanism on top of taxes, allowing nodes to choose where their taxes go in the prediction market.
Additionally, since Automated Market Makers are closely related to prediction markets, we can leverage many DeFi tools in this context.
Allow users to stake behind nodes. This is similar to prediction markets, but with slightly different economics. Like with prediction markets, it may be possible to tax users and then allow them to choose which nodes they stake behind. Overall, this approach is similar to delegated Proof-of-Stake.
Can a node announcing that it successfully cheated (and thereby receiving a reward) benefit the robustness of the protocol? How much would this node have to be rewarded?
How often should a client check results? Clearly it is related to the amount of collateral that the other node deposits, how much they value getting true/false results, reputation, and so on. This is a parameter that the client would need to learn to maximize its own utility.
The ledger can maintain a record, for each compute node, of the number of jobs the compute node has completed, the number of times its results were checked, and the number of times those results were replicated successfully. All other nodes (client and compute) could locally run some arbitrary function over these numbers to determine how reputable they find that node.
Results can only be replicated for as long as the inputs are stored somewhere. The client, compute node, or some other entity can pay for storing the inputs/outputs of jobs. The longer they are stored, the more time there is to check the results, which affects things like collateralization, the frequency of checks, etc.
This is related to, but not totally overlapping with, the amount of time that a node might have to wait before getting paid, which is the same time interval that a client has to check a result. However, checking the result after the node gets paid and receives back its collateral may still be possible, with other penalty schemes (or reward schemes, for example, coming from jackpots).
Colluding requires the following knowledge in order to enforce the parameters of the collusion.
The public keys of the nodes participating in collusion.
The results that were posted by those public keys.
The payouts to the public keys.
In order to sign a collusion contract to begin with, the public keys must be known. However, in a mediation protocol with enough nodes, it may be possible to obscure (2) and (3) by
Having nodes submit results to the mediation protocol in an obscured/anonymous way
Have nodes be paid out according to the results of the mediation protocol in an obscured/anonymous way
If these two criteria can be met, then a mediation protocol based on them might be capable of imitating the game-theoretic outcomes seen in the Smart Contract Counter-Collusion paper.
There have been many decades of cryptography and security research focusing on similar problems to these. It may be the case that it is already possible to do this; otherwise, there is a large amount of ongoing research on the topics of privacy-preserving transactions, and much prior work in the flavor of secret-sharing/MPC/Tor/Monero/ZKPs that could enable this.
These documents provide a background for ongoing Lilypad research.
These documents provide a background for the ongoing research on Lilypad. They are primarily focused on the game theory and cryptoeconomics of the protocol, and include an introduction to verifiable computing, an overview of the specific problems we are tackling, a brief overview of prior work, a description of our approach, and mechanisms that we plan to test in simulation.
Investigation of some previous verification-by-replication computing protocols
Before explaining our approach, we give a short overview of three prior approaches to verification-via-replication distributed computing protocols: Truebit, Modicum, and Smart Contract Counter-Collusion. We will outline potential improvements, and how our work builds on top of, and differs from, prior work.
Truebit is a protocol for outsourcing computation from blockchains, built using smart contracts on top of Ethereum. The original potential use cases were trustless mining pools, trustless bridges, scaling transaction throughput, and, scalable “on-chain” storage. Since its goal is to scale on-chain computation, it aims for global consensus: "Since there exist no trusted parties on Ethereum’s network, by symmetry we must allow any party to be hired to solve any computational task, and similarly anyone should be able to challenge a Solver’s outcome of a computational task. The latter requirement ensures that TrueBit operates by unanimous consensus." (emphasis added)
The components of Truebit are Miners, Task Givers, Solvers, Verifiers, and Judges. In order to incentivize checking results, random errors are forced into computations, with jackpots awarded to those who find them. These jackpots are funded by taxes on computations.
The authors claim that Sybil attacks are mitigated by pairwise Sybil-resistance between the parties of Task Givers, Solvers, and Verifiers, with Judges and Referees, whose roles are played by Miners, assumed to function as intended. Likewise, they claim that attacks to get bogus solutions on-chain by scaring off Verifiers are mitigated by the economics of deposits, taxes, and jackpot rewards. Additionally, a cartel of Solvers who absorb losses until they find a task with a forced error, upon which time they will receive the jackpot, will lose money in the long-term, since the expected Solver deposit per task is higher than the expected jackpot per task. Addressing another attack, the authors claim that an attack involving a flood of Challengers who try to take as much of the jackpot reward resulting from a forced error as possible is mitigated by having the total jackpot reward decrease as the number of Challengers increases.
Does not scale to large/complicated/arbitrary computations
No formal theorems or proofs, no simulations, many plausible but unsubstantiated claims, especially regarding collusion
Everything is done on-chain
This model does not work well with two-sided marketplaces, because
It aims for global consensus, where any node is allowed to do the computation, whereas in two-sided marketplaces, clients need to be able to choose which nodes they are paying to do the computation
Clients may have time restrictions on their computations, and cannot wait for cases where their computations were injected with forced errors
No accounting for repeated games
Taxes and jackpots are a valuable tool to create a global game that affects local outcomes
Provides a list of potential client attacks
Job Creators are only allowed to submit deterministic jobs to the protocol. The Mediator exists to check for non-deterministic tasks submitted by the Job Creator (which can be used by the Job Creator as an attack vector to get free results), and fake results returned by the Resource Provider. The method for determining whether a job is deterministic or not is for the Mediator to run a job n times and check to see whether it receives different answers.
Modicum combines two separate ideas: checking the result from a Resource Provider to see if it is correct, and checking a result from a job submitted by a Job Creator to see if the job was deterministic or not. This implies that there is no capability for a client to simply check whether a result is correct or not, without the possibility of its collateral being slashed.
An alternative to trusting the Mediator (to act as a TTP) by having it run a job n times is having a consortium of n Mediators each run the task a single time. However, this adds the complication of achieving consensus in that consortium.
The issue of the Resource Provider and Mediator colluding to return a fake result is not addressed by this protocol. The authors allow for a Job Creator or Resource Provider to remove a Mediator from their list of trusted Mediators if they no longer trust it. However, that still leaves room to cheat at least once, and ideally this should be disincentivized from the outset.
There is also the issue of collateralization. The Modicum protocol, as well as a number of other protocols, assume (approximate) guesses as to the cost of jobs in advance, so that nodes can deposit the correct amount of collateral. However, doing so is fraught with complications; we provide an alternative approach in the Mechanisms to Explore section.
The Mediator is basically a trusted third party
Client cannot simply check a result without being slashed, which is a consequence of the client attack model
No accounting for repeated games
Potential client attack, though one that can be mitigated by technical means
The client has benefit of getting correct results, which needs to be accounted for in simulation
The authors determine that cryptographic methods for verifiable computation are too expensive for real-world scenarios. For that reason, they rely on verification-via-replication. The scenario is one in which a client simultaneously outsources computation to two clouds, where those two clouds deposit collateral into smart contracts in such a way to create a game between them, where the game incentivizes doing the computation honestly. The central challenge that the authors tackle is the issue of collusion - that is, what if the two clouds collude on an incorrect answer?
In contrast to Modicum, the client is assumed to be honest, and in contrast to Truebit, a trusted third part (TTP) is used to handle disputes.
The authors use a series of three contracts to counter collusion.
The first game is an induced Prisoner's Dilemma - to avoid the two clouds colluding, one cloud can be rewarded the other cloud's deposit (minus payment to the TTP for resolving the dispute) if the former returned the correct result and the latter did not. Thus, each cloud is better off giving the other cloud fake results while computing the correct result itself. This contract is called the Prisoner's contract. It is analogous to the equilibrium in the classic prisoner's dilemma being defection <> computing honest result and giving other node fake result if offered to collude.
However, the clouds can agree to collude via a smart contract as well. They could do this by both depositing another amount into a contract, where the leader of the collusion must add a bribe (less than its cost of computing) to the contract as well (disregarding the bribe, both clouds deposit the same amount of collateral). The deposit is such that the clouds have more of an incentive to follow the collusion strategy than to deviate from it. This contract is called the Colluder's contract.
In order to counteract the Colluder's contract, a Traitor's contract is used to avoid this scenario by incentivizing the clouds to report the Colluder's contract. The basic concept is that the traitor cloud indeed reports the agreed-upon collusion result to the client in order to avoid the punishment in the Colluder's contract, but also honestly computes and returns the result to the client in order to avoid the punishment of the Prisoner's contract. The client must also put down a deposit in the Traitor's contract. Only the first cloud to report the Colluder's contract gets rewarded. The signing and reporting of the contracts must happen in a particular order in order for this process to work.
The authors prove that these games individually and together lead to a sequential equilibrium (which is stronger than a Nash equilibrium), meaning that it is optimal not only in terms of the whole game, but at every information set (basically the set of options each player has at every turn).
A Colluder's contract can be signed on different chains (or even off-chain). In order to mitigate this, the Traitor's contracts would have to become cross-chain (which is a major technical challenge), not to mention the possibility of cryptographically secure contracts (e.g. MPC contracts) where one of the parties alone would not be able to prove the existence of this contract
Relies on trusted third party to resolve disputes
Every task is replicated (that is, two copies of each job are always computed)
Assumes client is honest
Assumes amount of collateral known beforehand
No accounting for repeated games
It is well known that in the repeated Prisoner's dilemma, depending on the assumptions, cooperation becomes the equilibrium
The contracts and the payoffs that they induce offer a valuable toolbox to think about the problem of collusion
The contracts offer, in a restricted setting, an ironclad way (assuming the proofs are correct) of preventing collusion
Our Approach
A core assumption in much of game theory is that agents are utility-maximizing. That is, agents are completely rational actors, and are able to execute exactly the behavior that maximizes their return, however "return" is defined in the given context.
However, we know that in real life, humans are not completely rational, and are not capable of perfect execution of actions. In that light, how can we look at the game-theoretic approaches in the last section?
Either we can try to account for the irrational behavior of humans, or we can try to emulate the behavior of utility-maximizing agents. While there is a large amount of game-theoretic literature dedicated to the former, we opt for the latter for reasons that will become clear below.
Note that the actual environment of a two-sided marketplace for distributed computation is extremely complicated (e.g. the heterogeneity of hardware, types of computation, latencies and bandwidths, etc.). Any theoretical/analytic approach to the problem that is actually correct should also work in simulation, so we opt for a simulation-driven approach.
The way that we can emulate perfectly rational behavior is by training autonomous agents to act on behalf of their human owners in a utility-maximizing manner. At that point, the challenge is to design the global game to drive the probability of cheating to zero - ideally, to make it be equal to zero - which is no small feat in a trustless and permissionless environment. However, the simplifying assumption that we are in fact operating with utility-maximizing agents conceptually simplifies the problem immensely.
At a minimum, the action space for an autonomous agent representing a compute node should be to cheat or not to cheat, and to collude or not collude within a mediation protocol. The observable environment for nodes on the network should include all data stored on the blockchain - that is, the sequence of deals, results, and mediations - as well as the information in the orderbook. While the orderbook will be off-chain, we model in the digital twin the orderbook acting as a centralized, single source of truth that all agents have access to. In the long-term, nodes will have (potentially non-identical) local information regarding other job and resource offers on the network.
Further work may explore agents forming beliefs about the hardware and strategies of other agents, but that is beyond the scope of the current work.
We conclude with two "axioms" upon which we will base our simulations:
Every agent attempts to maximize its utility, including cheating and/or colluding if necessary.
All other components of the game should lead to a "good" solution, as defined in the problem statement.
Verifiable computing is dedicated to ensuring that outsourced computations (that is, computations that are not done locally) are done correctly. In some scenarios, it cannot be assumed that the node to which a job is being outsourced will compute the result honestly, or that it is not faulty. Moreover, verifying the result should have less overhead than computing the result in the first place.
While blockchains provide safety and liveness, the massive replication of computation becomes too costly when that level of security is not needed. There is a difference between global consensus, which is necessary in blockchain environments, and local consensus, which is more suited for two-sided marketplaces. In global consensus, all nodes need to be convinced that every computation was done correctly. In contrast, in local consensus, only a small number of nodes - potentially only one node, the client - needs to be convinced that a computation was done correctly.
Ostensibly, for a two-sided marketplace, this implies that only a client really needs to be convinced that a computation was done correctly. However, these computations are not done in isolation, and the interrelation between a client choosing one node repeatedly versus many different nodes, and the mathematics behind those decisions, as well as the need to create a protocol that any client can come along to with no prior experience and trust that cheating is disincentivized, implies the creation of a global game that, while not requiring global consensus in the traditional sense, emulates it in some manner.
One way to ensure that computations were done correctly is by using cryptographic methods. There are a number of cryptographic approaches for verifiable computation, including
Interactive Proof (IP)
In interactive proofs, verification of a statement is modeled as an interaction between a prover and a verifier. The goal of the prover is to convince the verifier that the statement is true, even when the verifier does not have the computation resources to do the computation itself.
The protocol must satisfy completeness (if the statement is true, an honest verifier will be convinced) and soundness (if the statement is false, the prover cannot convince the verifier except with some negligible probability).
Zero-Knowledge Proof (ZKP)
Zero-knowledge proofs are a type of interactive proof where the verifier learns nothing about private inputs of the computation, other than that the outputs were computed correctly from the all the inputs (some of which may be public/known to the verifier).
A ZKP can be made non-interactive, in which case it is called a Non-Interactive Zero-Knowledge Proof (NIZK). Two common variants of NIZKs are zk-SNARKs (zero-knowledge Succinct Non-interactive Argument of Knowledge) and zk-STARKs (zero-knowledge Scalable Transparent Argument of Knowledge).
Like IPs, ZKPs must also satisfy the requirements of completeness and soundness.
Multi-Party Computation (MPC)
Multi-party computation allows multiple parties to jointly compute a function over their individual inputs without any party revealing its input to other parties. The main objectives of MPC are privacy (parties should learn known about each others' inputs), security (some level of anti-collusion preventing malicious attempts to learn information), functionality (the ability to compute functions over data), and robustness (the protocol should work correctly even in the presence of malicious behavior or faults).
Trusted Execution Environments are secure and isolated enclaves, where code and data inside of the enclave are insulated from the rest of the system, including the operating system, applications, and other enclaves. The goal is to maintain both the confidentiality and the integrity of the code and data.
Verification-via-replication - often described using the adjective "optimistic" in the blockchain world - relies on recomputing the computation to check whether the end result is the same. The benefits of this method are that it is the easiest to understand, and in some sense, the easiest to implement.
In contrast to the other approaches, verification-via-replication often requires reliance on game-theoretic mechanisms such as collateral slashing, reputation, and other methods. This can become a bit complex when trying to counter collusion between the nodes that are computing the results.
One of the downsides of this approach is, of course, the extra effort expended on recomputing computations. However, with proper incentives, the overhead of this can be reduced dramatically. It is also important to keep in mind that the overhead of this approach is much lower than cryptographic methods, which usually have much higher overhead.
We opt for verification-via-replication as a first approach, for the reasons that it is simple to understand, has less overhead than cryptographic approaches, and has an attack surface that can be economically modelled and analyzed.
This has the downside of making private computations difficult. While the inputs and outputs of jobs can be encrypted so that only the client and compute node can see the code and data, this still leaves the client vulnerable to having their information leaked. Future approaches can incorporate SLAs and eventually support for homomorphic encryption to deal with this issue.
Instructions for setting up a Resource Provider (node) on the public Lilypad testnet using Docker, including obtaining necessary funds and installing required software.
Linux (Ubuntu 22.04 LTS)
Nvidia GPU
The testnet has a base currency of ETH, as well as a utility token called LP. Both are used for running nodes. To add a node to the testnet, follow these steps:
We recommend using MetaMask with custom settings to make things easier. Once you have it installed and setup, here are the settings you need to use:
The faucet will give you both ETH (to pay for gas) and LP (to stake and pay for jobs).
The Lilypad Network uses the Arbitrum Sepolia Testnet to settle compute transactions. When a transaction is ready to be saved on-chain, Lilypad cycles through a list of public Arbitrum Sepolia RPC endpoints using the endpoint that settles first to save the compute transaction.
Before starting, export your private key from MetaMask. Follow the official MetaMask guide for instructions on safely exporting your private key.
Use curl to download the docker-compose.yml file from the Lilypad GitHub repository.
If any containers named resource-provider, ipfs, or watchtower are already in use, they will need to be stopped before running this setup to avoid naming conflicts.
You can check if these containers are running with:
If they are running, stop them with:
If there are still conflicts when trying to running with the docker-compose file, remove the containers:
Start the Lilypad containers using Docker Compose:
To include a custom RPC URL:
You must not reuse your compute node key as a client, even for testing: this will result in failed jobs and will negatively impact your compute node since the wallet address is how nodes are identified on the network.
Use the following command to check the status of the resource provider and bacalhau.
Use the following command to view the containers running after starting Docker Compose.
When a new version of Lilypad is released, it is critical for Resource Providers to update their installations to ensure compatibility and ability to run Lilypad jobs.
To view your Lilybit_ rewards, visit one of the following dashboards and paste your node's public address into the input:
Setup a personal Arbitrum Sepolia RPC for a Lilypad Resource Provider.
Select the “free” tier as the compute units provided should be sufficient to run a Lilypad RP.
In the Infura dashboard, a new API key will usually generate automatically. If not, select "Create New API Key". Navigate to "Configure" to setup the API key.
Scroll down the list to the Arbitrum network and ensure the Sepolia testnet box is checked, then save changes.
In the API key dashboard, select "Active Endpoints" and navigate to "WebSockets".
Scroll down the page to find the Arbitrum Sepolia URL. The RPC endpoint for Arbitrum Sepolia is ready to be used with the Lilypad Resource Provider:
Lilypad RPs can use a personal RPC endpoint with a few simple steps. Only Web-socket (WSS) connections are supported.
Stop the existing Lilypad Resource Provider (RP) before setting up the new RPC.
Locate the Lilypad RP Docker container using:
Stop the container using the PID:
Use this command to start the lilypad-resource-provider.service with the new RPC:
Check the status of the container:
Stop the existing Lilypad RP (if the node is not running, disregard this first step):
Update lilypad-resource-provider.service with the new RPC:
Add following line to [Service] section:
Reboot the node:
Catch up with the Lilypad team or hack on the Lilypad Network at these events!
View the upcoming events we are attending or hosting.
Online
March 17th
Dubai
April 30th - May 1st
View the previous events we attended or hosted.
Denver
February 23rd - March 2nd
South East Asia
November 9th - November 17th
Brussels
July 8th - July 11th
Online
May 17th - June 7th
San Francisco
October 18th - October 20th
Bangkok
November 12th - November 15th
Sydney
May 3rd - May 5th
Istanbul
November 13th - November 17th
Iceland
September 25h - September 28th
Barcelona
October 1st - October 3rd
Las Vegas
October 3rd - October 5th
Online
September 23rd
Instructions for setting up a Resource Provider (node) on the public Lilypad testnet using Linux, including obtaining necessary funds, installing required software, and ensuring security measures.
Linux (Ubuntu 22.04 LTS)
Nvidia GPU
Nvidia Docker drivers
The testnet has a base currency of ETH, as well as a utility token called LP. Both are used for running nodes. To add a node to the testnet, follow these steps:
We recommend using MetaMask with custom settings to make things easier. Once you have it installed and setup, here are the settings you need to use:
The faucet will give you both ETH (to pay for gas) and LP (to stake and pay for jobs).
To set up your environment for using Lilypad with GPU support, you need to install several key components. This guide will walk you through installing Docker, the Nvidia Container Toolkit, Bacalhau, and Lilypad. You'll also configure systemd to manage these services efficiently.
Docker is a platform that allows you to automate the deployment of applications inside lightweight, portable containers.
To install Docker Engine, follow the steps specific to your operating system from the official Docker documentation:
Configure the container runtime by using the nvidia-ctk command:
The nvidia-ctk command modifies the /etc/docker/daemon.json file on the host. The file is updated so that Docker can use the NVIDIA Container Runtime.
Restart the Docker daemon:
Bacalhau is a peer-to-peer network of nodes that enables decentralized communication between computers. The network consists of two types of nodes, which can communicate with each other.
To install Bacalhau, run the following in a new terminal window (run each command one by one):
To check your Bacalhau version use:
The expected output is:
The installation process for the Lilypad CLI involves several automated steps to configure it for your specific system. Initially, the setup script identifies your computer's architecture and operating system to ensure compatibility. It will then download the latest production build of the Lilypad CLI directly from the official GitHub repository using curl and wget.
Once the CLI tool is downloaded, the script sets the necessary permissions to make the executable file runnable. It then moves the executable to a standard location in your system's path to allow it to be run from any terminal window.
Via official released binaries
To verify the installation, run lilypad in the terminal to display the version and a list of available commands, indicating that Lilypad CLI is ready to use.
You will need to create an environment directory for your node and add an environment file that contains your node's private key.
To do this, run the following in your terminal:
Next, add your node's private key into /app/lilypad/resource-provider-gpu.env:
This is the key where you will get paid in LP tokens for jobs run on the network.
You must not reuse your compute node key as a client, even for testing: this will result in failed jobs and will negatively impact your compute node since the wallet address is how nodes are identified on the network.
The Lilypad Network uses the Arbitrum Sepolia Testnet to settle compute transactions. When a transaction is ready to be saved on-chain, Lilypad cycles through a list of public Arbitrum Sepolia RPC endpoints using the endpoint that settles first to save the compute transaction.
systemd is a system and service manager for Linux operating systems. systemd operates as a central point of control for various aspects of system management, offering features like parallelization of service startup, dependency-based service management, process supervision, and more.
To install systemd, open /etc/systemd/system/bacalhau.service in your preferred editor:
Open /etc/systemd/system/lilypad-resource-provider.service in your preferred editor.
Reload systemd's units/daemons (you will need to do this again if you ever change the systemd unit files that we wrote, above)
Start systemd units:
Now that your services have been installed and enabled, check the status of Bacalhau to ensure it is running correctly on your node:
To check if the node is running use the following command:
This will give a live output from the Lilypad node. The logs will show the node running and accepting jobs on the network.
Run the following command to get more status info from your node:
To restart your resource provider run:
Without a discussion opened, our team will not be able to support the problem.
Description (including Lilypad version running on your node)
Hardware Info (including Linux/Windows version)
Related blockchain/ETH addresses of transaction hashes
Output Logs - sudo systemctl status lilypad-resource-provider
Related links/urls
Screenshots
When a new version of Lilypad is released, it is important for resource providers to update their installations to ensure compatibility and access to the latest features and improvements.
Please note that using sudo rm -rf is very powerful and can be dangerous if not used carefully.
If the Lilypad RP is running, stop the system (if the node is not running, disregard this first step):
Remove the Lilypad executable by running:
Start your resource provider by running:
To disconnect your node from Lilypad you will need to do a few things to completely offboard.
Using sudo rm -rf is very powerful and can be dangerous if not used carefully. It is highly recommended to navigate to the parent directory and remove files from there to avoid accidental deletion of important files.
First, stop the node:
Next, you must remove the .service files related to Lilypad and Bacalhau. These files are typically stored in /etc/systemd/system/. To remove them, run the following command:
Next we notify the systemd manager to reload its configuration by running:
Then, remove the environment file for the Lilypad resource provider. This file is usually stored in /app/lilypad/. To remove it, run:
Finally, if you followed the installation instructions from the Lilypad documentation and moved the executable to /usr/local/bin/lilypad, it can be removed from there. If the executable is stored in a different directory on your machine, navigate to that directory and remove it from there. To remove the executable, run:
To remove Bacalhau, run:
To view your Lilybit_ rewards, visit one of the following dashboards and paste your node's public address into the input:
If you want to allowlist only certain modules (e.g. Stable Diffusion modules), to control exactly what code runs on specific nodes (which can be audited to ensure that they are secure and will have no negative impact on the nodes), set an environment variable OFFER_MODULES in the GPU provider to a comma separated list of module names, e.g. sdxl:v0.9-lilypad1,stable-diffusion:v0.0.1.
Frequently Asked Questions for Lilypad Network
Lilypad is developing a serverless, distributed compute network that enables internet-scale data processing for AI, ML & other arbitrary computation from blockchains, while unleashing idle processing power & unlocking a new marketplace for compute.
Lilypad provides decentralized AI computational services. By extending unrestricted, global access to computational power, Lilypad strategically collaborates with decentralized infrastructure networks, such as Filecoin, to formulate a transparent, efficient, and accessible computational ecosystem.
Perform off-chain decentralized compute over data, with on-chain guarantees, and to call this functionality directly from a smart contract, CLI and an easy to use abstraction layer, opens the door to a multitude of possible applications.
Bacalhau is a peer to peer computation network enabling compute over data jobs like GPU-enabled AI, ML, analytics, data engineering, data science, de-sci and more. With the open-source Bacalhau Project, you can streamline your existing workflows without rewriting by running Docker containers and WebAssembly (WASM) images as tasks. This architecture is also referred to as Compute Over Data (or CoD).
Previously, Lilypad required deterministic jobs on the network and used optimistic reproducibility to randomly re-run jobs to ensure trust, however this method has been deprecated due to:
The limitation the determinism requirement placed on what jobs were available to run on the network
The inability to realistically be able to determine a “deterministic” job to be deterministic easily
Yes, Lilypad closed our seed round of funding in March 2024.
The Lilypad Incentivized testnet launched in mid June 2024.
Lilybit_ rewards will be awarded to nodes for time on the network (up to a 4x multiplier) and compute power brought to the network. Rewards will be redeemable, for the Lilypad ERC20 Utility Token at Mainnet Launch, with between 5% and 10% of the total token supply (depending on IncentiveNet demand and tokenomics finalization) being allocated to this phase of the Lilypad Network.
Phase 1 of the Incentivized Testnet is focused on rewarding nodes on the network, referred to as Resource Providers. The earlier a provider joins the network, the more Lilybits_ will be available.
Phases 2 and onward will provide rewards for Lilypad modules created as well as developer tools/apps (in addition to rewarding nodes).
You can check your rewards by pasting your nodes wallet address into the following interfaces:
On the Lilypad network, The blockchain is used for
Payment rails
Storing the deals transparently (on-chain guarantees about the compute)
Storing any disputes & results
Lilypad Tokens are used to transact on the Lilypad network. They are used as payment by those who want to run jobs (to the resource providers who run them), and as collateral by resource providers.
You need ETH tokens to pay for the gas fees for smart contracts the facilitate transactions, and for records of transactions and disputes that are posted to the Ethereum blockchain.
The required steps before running a Lilypad node include adding the node, adding the Lilypad network information, obtaining tokens and installing the required software.
Linux (latest Ubuntu LTS recommended)
Nvidia GPU
Nvidia drivers
Docker
Nvidia docker drivers
Although the launch date is not finalized, the launch of Lilypad Mainnet and the TGE is scheduled for 2025.
If you have questions or find something in here interesting, please feel free to raise a discussion in the or in !
The verification game consists of a series of rounds, where in each round, a smaller and smaller subset of the computation is checked. Eventually, only one instruction is used to determine whether a Solver or Verifier is correct:
The original version of had five key components: Job Creators (JC), Resource Providers (RP), Solvers (market makers), Mediators (agreed-upon third parties for mediation), and Directories (file systems, which we have replaced with IPFS and Docker registries). Job Creators are clients, the ones who have computations that need to be done and are willing to pay. Resource Providers are those with computational resources that they are willing to rent out for the right price. Solvers are market-makers; they match the offers from JCs and RPs. Mediators are third parties trusted by both JCs and RPs to arbitrate disagreements. The Directories are network storage services available to both JCs and RPs.
Useful prototype for a two-sided marketplace (the for stream processing applications is also useful)
While this problem setting - verifiable computation by way of game theory - is different than many game theoretic settings, we can draw inspiration from commonly used concepts like the and . Both strategyproofness and the revelation principle are centered around the idea of incentivizing agents to truthfully report their preferences. Most approaches in the literature rely on analytic methods to determine what rational agents will do by analyzing their payoffs as a function of their preferences, the behaviors of other agents, and the mechanism under analysis. Ultimately, we are also aiming to find (a) mechanism(s) that lead(s) to an equilibrium where all agents choose to not cheat and not collude.
The process begins by creating a digital twin of a two-sided marketplace. In this environment, autonomous agents acting on behalf of client and compute nodes will be trained to maximize returns based on data gathered in simulation. For now, we will avoid maximizing returns by optimizing scheduling, though this is a future topic of interest. We will use techniques primarily from the field of multi-agent reinforcement learning in order to train the agents. The precise methods we will use (e.g. modes of training and execution, homogeneous vs. heterogeneous agents, choice of equilibrium, self-play vs. mixed-play, value-based vs. policy-based learning, etc.) will be determined in the course of building the simulation. See the by Albrecht, Christianos, and Schäfer for our reference text.
(Ubuntu install)
(NVIDIA Container Toolkit)
For a more in-depth look at the requirements to run a Lilypad node, please refer to the documentation.
New RPC URL:
For a step by step guide on adding the network, please refer to our .
To obtain testnet LP, use the and enter your ETH address.
To obtain testnet ETH, use a third party and enter your ETH address.
If you need more tokens and already have Sepolia ETH, use the to transfer tokens over to Arbitrum Sepolia.
Resource Providers have the option to using Alchemy instead of using the default public RPC endpoints.
A personal RPC endpoint helps RPs to avoid reliability issues with the public RPC endpoints used by Lilypad ensuring rewards can be earned and jobs can be run consistently. RPs running a personal RPC endpoint contribute to the fault tolerance and decentralization of the Lilypad Network! Read more in the Alchemy Arbitrum .
Before we start the Rp with the Docker setup, retrieve the private key from the wallet set up earlier in this guide. For guidance on exporting your private key, refer to . Once the private key has been retrieved, proceed to initialize the Docker containers using the commands provided below.
The same WEB3_PRIVATE_KEY cannot be used for both a RP and using the Lilypad CLI. If a WEB3_PRIVATE_KEY has already been used to run jobs with the CLI, make a new one and . You can `unset $WEB3_PRIVATE_KEY` if you want to use a different one to run your RP.
Before moving to the next step, ensure older versions of lilypad and bacalhau as systemd services.
Using the Lilypad Docker setup a new RP and run.
are some common troubleshooting techniques when it comes to your resource provider using Docker.
This personal RPC endpoint allows Resource Providers (RP) to avoid reliability issues with the RPC endpoints used by Lilypad ensuring rewards can be earned and jobs can be run consistently. RPs running a personal RPC endpoint contribute to the fault tolerance and decentralization of the Lilypad Network! Read more in the Alchemy Arbitrum .
This guide walks through the steps of setting up a personal RPC endpoint for Arbitrum Sepolia using .
on Infura and choose your plan based on how many APIs you need.
This is guide is for individuals running a Lilypad Resource provider, find more info .
If the Lilypad RP was properly as a systemd service, the RP will reboot using the new RPC. Once the reboot is complete, the RP should be running with the updated configuration. To verify your node is back online and running correctly, run the following:
These instructions are no longer actively maintained. We recommend using the for the most up-to-date and supported setup.
For a more in-depth look at the requirements to run a Lilypad node, please refer to the documentation.
(Ubuntu install)
New RPC URL:
For a step by step guide on adding the network and importing the LP testnet token, please refer to our .
To obtain testnet LP, use the and enter your ETH address.
To obtain testnet ETH, use a third party and enter your ETH address.
The Arbitrum Sepolia faucet provides 0.0001 tokens per request. If you need more tokens and already have Sepolia ETH, you can use the to transfer it over to Arbitrum Sepolia.
- Docker Engine
To ensure proper operation of your graphics cards and Lilypad, follow these steps to install the Nvidia Toolkit Base Installer:
If the Bacalhau CLIENT version is not v1.6.0, it will need to be replaced. Follow the steps to uninstall and reinstall Bacalhau.
A can be retrieved from the Metamask account details menu. For more info, check out official guide from Metamask on viewing a wallet's private key.
Resource Providers have the option to using Alchemy instead of using the default public RPC endpoints.
A personal RPC endpoint helps RPs to avoid reliability issues with the public RPC endpoints used by Lilypad ensuring rewards can be earned and jobs can be run consistently. RPs running a personal RPC endpoint contribute to the fault tolerance and decentralization of the Lilypad Network! Read more in the Alchemy Arbitrum .
For complex issues, bug reports, or feature requests, open a discussion in the Lilypad-Tech Github organization discussion .
Navigate to the discussion , select "New Discussion", choose "rp-issues", and fill out the template.
For quick questions or minor issues, use the Lilypad Discord channel and provide the following info.
As every system is different, these instructions may vary. If you have any issues, please reach out to the team in the for help!
Visit the for a full list of available modules.
The Lilypad Whitepaper was planned to release by end of Q4 2024, but required further review from our team and advisors. See our for the current timeline.
Find the full Lilypad Network on our website!
To find out more about it, see the
Resource Providers (RP) can track their Lilybit_ earnings with the .
Refer to the documentation and select your preferred platform (Linux or Docker) for a detailed guide on the prerequisites.
For more information, please visit .
Resource providers are expected to have the latest Lilypad versions installed on their machines. These will be included in the Docker files in each release. For more information, please refer to the .
Find more information check out the docs.
View the .
Follow the to get started.
To build an application with Lilypad compute and modules on the backend, check out this .
A Lilypad module is a Git repository that can be used to perform various tasks using predefined templates and inputs. This will walk you through the process of creating a Lilypad module, including defining a JSON template, handling inputs, ensuring determinism, and other best practices.
Lilypad is an open network that allows anyone to contribute GPU computing capacity. Find instructions for running a node in our documentation.
For more information on the requirements to run a Lilypad node, please refer to the documentation.
Get started with Lilypad
Lilypad allows you to run on-demand compute jobs on the network, enabling AI inference and high-performance workloads without managing infrastructure.
Setup a personal Arbitrum Sepolia RPC for a Lilypad Resource Provider.
Select the “free” tier as the compute units provided should be sufficient to run a Lilypad RP. The free service provides 300 million compute units per month.
Select “skip bonus” or input a credit card with billing info (the card will not be charged unless the compute credits in the free tier are used).
In the “Overview” section of the Alchemy dashboard, navigate to “My app” and select “Endpoints”. If an app was not created upon login, create a new one by selecting "Create new app".
By selecting “Endpoints”, the “Networks” tab will open providing an option to configure the Arbitrum API.
Select “Sepolia”
Select “Websockets”
The RPC endpoint for Arbitrum Sepolia is ready to be used with the Lilypad Resource Provider:
Metrics for the RPC can be viewed in the “Metrics” tab.
Lilypad RPs can use a personal RPC endpoint with a few simple steps. Only Web-socket (WSS) connections are supported.
Stop the existing Lilypad Resource Provider (RP) before setting up the new RPC.
Locate the Lilypad RP Docker container using:
Stop the container using the PID:
Use this command to start the lilypad-resource-provider.service with the new RPC:
Check the status of the container:
Stop the existing Lilypad RP (if the node is not running, disregard this first step):
Update lilypad-resource-provider.service with the new RPC:
Add following line to [Service] section:
Reboot the node:
This guide walks through the steps of setting up a personal RPC endpoint for Arbitrum Sepolia using .
and login to the Alchemy dashboard.
This is guide is for individuals running a Lilypad Resource provider, find more info .
If the Lilypad RP was properly as a systemd service, the RP will reboot using the new RPC. Once the reboot is complete, the RP should be running with the updated configuration. To verify your node is back online and running correctly, run the following:
Alison Haire
CEO
prev: Filecoin, IBM, Accenture
WebSearch UI
Combines websearch and AI powered answers using the Lilypad Inference API to help answer questions based on the latest happenings provided as context.
DefiKicks
An innovative NFT game where characters evolve and improve their tasks over time.
SlickCI One click deployment of CI/CD workflows on to decentralised cloud infrastructure.
dub3.ai dub3.ai is an innovative marketplace harnessing the power of Cartesi, enabling users to transform their voice into unique NFTs and generate text-to-speech content using cutting-edge AI technology.
A custom lilypad module for SQL streaming through kamu.dev, this provides provenance, version control, and provenance of derivative datasets. Integrates the desci project kamu.dev into a bacalhau job to be consumed by lilypad v2.
Dive into the cosmic engineering that propels your personalized galaxy of knowledge.
ML Tooling for Security Audits. Validate and test the status of deployed Lilypad Compute Networks. The contracts ensure seamless interaction with the network, while the test script checks for network availability and contract functionality.
This platform, much like Docker Hub, is a one-stop destination for all your module needs as a Lilypad user. Includes a feature-rich playground where you can seamlessly test job specifications, experiment and fine-tune your code in a safe environment.
A user can plug obsidian-lilypad into their obsidian instance to fetch dpids from desci labs, have chat gpt summarize it, and use that as a prompt in runSDXL in lilypad. Helia/ipfs and kubo delegated content routing is used to splay dags and fetch content from IPFS.
The core objective of DeCenter AI is to democratize and decentralize AI model training.
Lilylatte is a DataDAO designed to bridge the gap between onchain and offchain data, focusing on web3 market research. Data owners, or web3 citizens, link their wallets and engage in dialogues generated by our LLM model, 'Lily.'
An intents computational framework for describing lilypad workflows with a serialization format that allows you to replace variables within solidity, onchain which allows you to configure the receiveResults to feedback and trigger future jobs.
I use historical data about the network gathered by Saturn Moonlet to prepare and publish analytical reports on Saturn Observatory website every month.
RejuvenateAI is the first community blockchain based project powered by community built for community to promote healthy living and achieve healthy locations where people live up to a 100 years all around the world.
This project provides a JavaScript wrapper to interact with the Lilypad CLI using Node.js. The wrapper utilizes the node-pty package to spawn a pseudo-terminal for executing the Lilypad CLI commands. It will enable the front-end developers to integrate Lilypad with other tools.
LilyWrite combines blockchain technology and artistic expression, enabling users to craft and own unique poetic artworks. Users generate poems using credits, represented by LW tokens, and acquire them as distinctive non-fungible tokens (NFTs), each symbolizing a unique blend of creativity and expression.
GreenBadge is an innovative on-chain funding platform dedicated to fostering sustainability and a brighter future for our planet. Within our platform, organizations committed to these crucial goals will gain access to a vast network of donors and like-minded brands who share a passionate vision for the future of our environment.
CLI Package for running 3D surface reconstruction with decentralized compute.
Empowering Patients with ownership of their electronic health records.
A decentralized credit score system using zkSNARKs, decentralized Computation & Storage.
Elevating financial privacy in crypto withdrawals. Securely withdraw funds via anonymous transfers & secure transactions, preserving anonymity.
Verifiable on-chain Large Language Models drift benchmarking inspired by
Uncensored 100% decentralized GPT AI chat running on the blockchain.
Trusted audio in the age of deepfakes 🔒🎙 Generative AI is a threat to society. It enables disinformation, manipulation, and political subversion. We've built the world's first attested microphone and used ZK-SNARKs to protect authenticity and privacy.
A fastchat LLM Inference Module for Lilypad v1.
Decentralized AI Assistant for Web3 - Perform on-chain transactions via prompt.
Private-Model Generation & Service Platform -> Your very own unique personality in an AI Avatar (web3 character.ai)!
Based on academic research, EasyCraft built an AI and blockchain based supply chain management implementation matching factory capacity with customer orders for economic efficiency.
A decentralized, community-governed Data DAO on Filecoin that democratizes DeFi data aggregation and TVL calculations
A decentralized, community-governed Data DAO on Filecoin that democratizes DeFi data aggregation and TVL calculations with on-chain adapter proposals, off-chain voting & interoperable tokens.
A proof of concept of a decentralized yield data aggregator. It gets the pool data, does calculations on it and shows the best APY value of the reserve to the user with a push notification.
Pensieve is a decentralized file storage to recreate your those moments in your memories and share with others. As you upload these memories, you can also choose to take part in the documentation of the HISTORY OF APES (HOMO SAPIENS).
A swiss army knife for Bacalhau, everything you need in an easily accessible dashboard.
An easy SDK for AI models running on decentralised computing.
