Image: > lilypad run sdxl:v0.9-lilypad3 "A record player at a bit event in space"

Upcoming

2024

• Devcon South East Asia [Nov 9-17]

2025

• TOKEN2049 Dubai [April 30 - May 1]

• ETHDenver [February 23 - March 2]

Past

• ETHCC Brussels [July 8-11]

• HackFS Online [May 17-June 7]

• ETH Global San Francisco [October 18-20]

• Devcon Bangkok [November 12-15]

• ETHGlobal Sydney [May 3-5]

• LabWeek Istanbul [Nov 13-17]

• Fil:Dev Iceland [Sept 25-28]

• Chainlink SmartCon Barcelona [Oct 1-3]

• Fil:Vegas [Oct 3-5]

• Open Data Hack - Online [Sept '23]

The installation process for the Lilypad CLI involves several automated steps to configure it for your specific system. Initially, the setup script identifies your computer's architecture and operating system to ensure compatibility. It will then download the latest production build of the Lilypad CLI directly from the official GitHub repository using curl and wget.

Once the CLI tool is downloaded, the script sets the necessary permissions to make the executable file runnable. It then moves the executable to a standard location in your system's path to allow it to be run from any terminal window.

Install via officially released binaries

Lilypad offers two distinct installation options to cater to different roles within the network: one for the users of Lilypad and another for resource providers who supply the computational resources to the Lilypad Network.

Select the appropriate installation based on your role:

# Detect your machine's architecture and set it as $OSARCH
OSARCH=$(uname -m | awk '{if ($0 ~ /arm64|aarch64/) print "arm64"; else if ($0 ~ /x86_64|amd64/) print "amd64"; else print "unsupported_arch"}') && export OSARCH;
# Detect your operating system and set it as $OSNAME
OSNAME=$(uname -s | awk '{if ($1 == "Darwin") print "darwin"; else if ($1 == "Linux") print "linux"; else print "unsupported_os"}') && export OSNAME;
# Download the latest production build
curl https://api.github.com/repos/lilypad-tech/lilypad/releases/latest | grep "browser_download_url.*lilypad-$OSNAME-$OSARCH-cpu" | cut -d : -f 2,3 | tr -d \" | wget -qi - -O lilypad

# Make Lilypad executable and install it
chmod +x lilypad
sudo mv lilypad /usr/local/bin/lilypad

# Detect your machine's architecture and set it as $OSARCH
OSARCH=$(uname -m | awk '{if ($0 ~ /arm64|aarch64/) print "arm64"; else if ($0 ~ /x86_64|amd64/) print "amd64"; else print "unsupported_arch"}') && export OSARCH;
# Detect your operating system and set it as $OSNAME
OSNAME=$(uname -s | awk '{if ($1 == "Darwin") print "darwin"; else if ($1 == "Linux") print "linux"; else print "unsupported_os"}') && export OSNAME;
# Download the latest production build
curl https://api.github.com/repos/lilypad-tech/lilypad/releases/latest | grep "browser_download_url.*lilypad-$OSNAME-$OSARCH-gpu" | cut -d : -f 2,3 | tr -d \" | wget -qi - -O lilypad

# Make Lilypad executable and install it
chmod +x lilypad
sudo mv lilypad /usr/local/bin/lilypad

Set WEB3_PRIVATE_KEY

You're required to set your private key environment variable, WEB3_PRIVATE_KEY, to interact securely with the network.

export WEB3_PRIVATE_KEY=<your private key>

To use the Lilypad CLI, the set private key will need to hold Lilypad testnet tokens and Arbitrum Sepolia ETH. You can find those instructions in the Funding your wallet documentation.

Verify installation

To verify the installation, running lilypad in the terminal should display a list of available commands, indicating that Lilypad CLI is ready to use.

Lilypad: <VERSION>
Commit: <COMMIT>

Usage:
  lilypad [command]

Available Commands:
  completion        Generate the autocompletion script for the specified shell
  help              Help about any command
  jobcreator        Start the lilypad job creator service.
  mediator          Start the lilypad mediator service.
  pow-signal        Send a pow signal to smart contract.
  resource-provider Start the lilypad resource-provider service.
  run               Run a job on the Lilypad network.
  solver            Start the lilypad solver service.
  version           Get the lilypad version

Flags:
  -h, --help             help for lilypad
  -n, --network string   Sets a target network configuration (default "testnet")

Use "lilypad [command] --help" for more information about a command.

Thats it! You've successfully installed the Lilypad CLI on your machine! 🎉

Introduction

Lilypad is a verifiable trustless decentralized compute network that aims to prevent cheating in the network. The network consists of clients and compute nodes. The main goal is to establish a game theoretic approach to verifiable computing, where clients can trust the results they receive from compute nodes. The approach used in Lilypad is pure verification by replication, without relying on cryptographic tools like snarks or trusted execution environments.

Global Consensus vs Local Consensus

In the context of Lilypad, it's important to understand the difference between global consensus and local consensus. Global consensus, as seen in blockchains, ensures that every node knows the result of a computation is correct due to massive replication across many nodes. However, in Lilypad's two-sided marketplace, only the client needs to be convinced that the result is correct. Local consensus is sufficient for the client, while other nodes like verifying nodes, requesters, and market makers may also need to be convinced.

Adversary Model

Lilypad assumes non-malicious but utility-maximizing agents, including both clients and compute nodes. Utility-maximizing means that nodes will do anything to maximize their return, including returning false results if necessary. However, they are not malicious and won't purposely send back false results. The adversary model assumes that all nodes are utility-maximizing and does not consider malicious behavior.

Good Solutions

Lilypad aims to achieve the best outcome, where nodes never have any incentive to cheat. If that's not possible, the goal is to minimize the incentive to cheat as a function of the protocol's parameters. Another option is to achieve the first two outcomes under simplifying assumptions, such as having a fraction of honest nodes in every mediation protocol. Lilypad also considers the possibility of not being able to achieve any of these outcomes and aims to understand the limitations.

Reinforcement Learning Approach

Lilypad takes an adversary-first approach by designing the most powerful adversary possible and then optimizing against it. The team uses multi-agent reinforcement learning to train agents to act as utility-maximizing agents on behalf of clients and compute nodes. Reinforcement learning has shown impressive results in various domains, and Lilypad aims to leverage its capabilities to train agents that can maximize their utility in the network.

Anti-Cheating Mechanisms

Lilypad plans to test various anti-cheating mechanisms once the reinforcement learning agents are implemented. These mechanisms include:

1. Consortium of mediators: A group of trusted mediators can check the results, reducing the likelihood of cheating.

2. Prediction markets and staking: Nodes can stake behind other nodes and lose their collateral if the node they stake behind is found to have cheated.

3. Taxes and jackpots: A tax is imposed on every job, and the taxes go into a jackpot that is awarded to nodes that find other nodes to have cheated.

4. Reputation: Nodes can build up a reputation based on the number of jobs they've done and the honesty of their results.

5. Sorting inputs and outputs: Storing inputs and outputs for longer periods allows for easier verification but increases collateralization requirements.

6. Frequency of checking results: Autonomous agents can decide when to check results, balancing the need for verification with the cost of collateralization.

Future Developments

In the future, Lilypad aims to take into account the preferences of nodes, such as computational requirements, time requirements, and scheduling requirements. The long-term goal is to have compute nodes and clients negotiate with each other automatically over these aspects of a job. Lilypad acknowledges that game-theoretic verifiable computing is a less studied form of verifiable computing compared to zero-knowledge proofs and trusted execution environments. However, the team is committed to exploring this approach and conducting rigorous research to find effective solutions.

Conclusion

Lilypad's game-theoretic approach to verifiable computing aims to prevent cheating in a decentralized compute network. By using reinforcement learning and testing various anti-cheating mechanisms, Lilypad strives to create a trustless environment where clients can have confidence in the results they receive from compute nodes. The team is actively working on implementing reinforcement learning agents and conducting simulations to evaluate the effectiveness of different strategies.

Problem Setup

The setup is a trustless, permissionless, two-sided marketplace for compute, where clients can purchase compute services from compute nodes. Trustless means that by default, the protocol does not assume that any particular node behaves in a trustworthy manner and that each node should be considered as rationally self-interested (note that this excludes intentionally malicious behavior). Permissionless means that any node can join or leave the network at will.

Matches for compute jobs are made off-chain, with the resulting deals and results recorded on-chain. Both clients and compute nodes need to agree to matches before they become deals, and make deposits to the protocol to enable rewards and punishments. Results are verified using verification-via-replication, and clients can check the results of any job after it has been completed, but before it needs to pay. It does so by calling upon a mediation protocol. The mediation protocol is the ultimate source of truth, and the outcome of the mediation protocol determines how payouts to nodes are made.

The issue of preventing fake results in the presence of a Trusted Third Party (TTP) as a mediator is effectively a solved problem (for example, see the section on prior verification-via-replication protocols, though there is much more literature on this topic). Given the assumption that the mediation protocol is the source of truth, we can treat the mediation protocol as a TTP. Since the fake results problem is basically already solved in this case, the cheating problem reduces down to solving the collusion problem within the mediation protocol. (Note, however, that we will address both cheating and collusion; the framework described here exists to conceptually simplify the problem.)

This is a typical scenario of a Byzantine environment, and we can use well-established approaches to Byzantine Fault Tolerance when invoking mediation. However, most BFT algorithms and cryptographic methods rely on assumptions regarding some fraction of honest nodes. The problem is that rational, utility-maximizing agents may still collude, even in a mediation consortium, in order to converge on incorrect results. On the one hand, we could assume that some fraction of nodes are honest, as is often done. On the other hand, can we do better?

Problem Statement

Task

The task is to find the mechanisms that incentivizes all nodes to behave honestly.

Adversary model

All agents in the protocol are utility-maximizing. This will be elucidated in a subsequent section. Most of the literature has focused on the case where compute nodes are dishonest. However, the client can also behave dishonestly in some manner that maximizes their utility. For example, if the client has some level of control over the choice of mediator, and dishonest nodes have their collateral slashed, the client could collude with the mediator in order to deem a correct result incorrect and get a cut of the honest compute node's collateral.

What is a good solution?

"Good" solutions can take a number of forms:

1. Nodes never have an incentive to be dishonest.

2. Nodes have an incentive to be dishonest that goes to zero as a function of the parameters of the protocol.

3. (1) or (2), but under some simplifying assumptions, such as there being some fraction of honest nodes within every mediation protocol.

A good solution would achieve any of these goals. Alternatively, another valuable outcome of this research would be to discover under what assumptions these goals can or cannot be met, or if the goals are even possible to achieve at all.

Mechanisms for achieving these goals

There are a number of ways that these goals may be achieved. The approach will be to construct a digital twin of the protocol and test a number of different mechanisms in simulation. These mechanisms include variations on mediation protocols, collateralization, staking, taxes and jackpots, and others; see the Mechanisms to Explore section for details.

AI Resources

• RAG vs Finetuning

• The promise and challenges of crypto + AI - Vitalik Buterin

• Data on Notable AI Models

• Towards decentralized AI

• Understanding the Intersection of Crypto and AI - Galaxy Digital Research

GPU Info

• How do graphics cards work?

• Nvidia CUDA Toolkit

• Nvidia CUDA in 100 Seconds

Bacalhau Resources

• Bacalhau.org

• Impact of Compute over Data - Juan Benet

• CoD vision & goals - Juan Benet

• Lessons for building Distributed Compute - Juan Benet & Molly Mackinlay

• State of CoD 2023 - David Aronchick

• Containers at the Edge - David Aronchick

Hackathon & Project Ideas

🤝 Crossover POCs, Integrations & Plugins!

Lilypad is aiming to build out a full suite of decentralized AI projects, which means we want to collaborate with other projects in the ecosystem to make this vision a reality.

These include linking Lilypad compute with storage providers, data streams, databases & more.

This also means adding layers like privacy or extending usage into other frameworks like Unity or React Native projects. Some examples include:

• Using Filecoin aggregators like Banyan or Tableland Basin with Lilypad

• Extending the Chainsafe Unity SDK to add Lilypad jobs as an option

• Building Lilypad into Mona or creating a module to use on Mona

• Integrating with The Graph

• Doing a POC with data from someone like WeatherXM

• Adding privacy layers for data with Lit Protocol

• ZK Computations using zkSync for verifiable credentials and identity proofs without revealing sensitive information

• A decentralized oracle service that uses AI to fetch, verify, and deliver real-world data from tools like Chainlink to smart contracts

There are also opportunities to extend the functionality of Lilypad compute with items like:

• CI/CD pipelines

• Vector databases

• Autonomous agents

• Batching jobs with an external script

• Chaining jobs together in a pipeline

💅 Contribute a Module

Contributing a module is one of the coolest things to do for Lilypad. There are so many many options for compute and AI jobs that you can run with Lilypad, including adding new state-of-the-art OSS AI models

• Text to speech

• Text to opera (have you heard this one!)

• Finetuning models for SDXL and LLM

• Image -> cartoon (or anything else)

• Build a module that classifies images into predefined categories (e.g., cats, dogs)

• A basic recommendation system for movies, books, or products based on user preferences

• Personal finance tracker that uses AI to categorize and analyze spending patterns.

• A module that provides personalized tutoring and learning assistance using AI

As a quick refresher, the current module making pipeline (see full "Build a job module" resource) looks like this:

1. Find or build a compute script (for example, a Python script on Hugging Face like SDXL)

2. Containerise the script with Docker (see this blog for more info)

3. Add a Lilypad Spec - this is simply a GitHub repo - you can grab a template here!

✨ DX Enhancements

Lilypad is still evolving and is currently in Beta phase. This means there are A LOT of opportunities to help build out better DX for your fellow developers!

One of the main opportunities is in the module making pipeline. See #contribute-a-module for a quick breakdown of the developer journey). Some of the opportunities to improve this process include:

• 💛 An application that can auto-containerise (or auto-dockerise) a given compute script

  • ❤️ BONUS IDEA: You could even then make a module to run the auto-containerising script on Lilypad after it's built! 🧙

• 💚 You could go one further and also have it auto-generate the Lilypad spec as well!

• 💛💛 Ways to make it easy for folks without GPUs at home to build & test a module

• 💚 Implement developer tooling for debugging and monitoring compute jobs, providing real-time feedback and logs

• 💛 Tools that analyze and optimize the performance of compute scripts, suggesting improvements and best practices

• 💚 Implement static analysis tools to check for code quality, style, and potential issues in compute scripts

• ❤️ Create tools to automatically audit compute scripts and modules for security vulnerabilities and compliance with best practices

👾 End User Projects

Build out other end-user projects across a broad range of verticals. We are very interested to hear the type of projects people are looking to build on the Lilypad Network and have released a Javascript wrapper to help folks build easily.

Here are a few basic ideas of end user projects:

• Education: AI-powered educational tools for personalized learning experiences

• Music: Developing AI-driven music composition tools, or applications for real-time music analysis and recommendation

• Finance: Leverage AI for risk assessment and trading strategies

• Retail: Building recommendation engines using AI for e-commerce platforms

• Environment: Developing solutions for environmental monitoring and pollution tracking

• Security: Tools using AI for auditing and security analysis

Projects across DeFi, DeSci, Gaming, & Metaverse, NFTs, IOT and more, as well #crossover-pocs-integrations-and-plugins!

🐐 Need more inspiration?

See the Past Hackathon Winners page!

✨We would love to hear your feedback!✨

We hope this page inspires you to build amazing things with us! If you have any ideas, suggestions, or any questions, we'd love to hear from you! Join our community on Discord to share your thoughts, ideas, ask questions, and get the support you need from the Lilypad team. 🪷

Join our Discord channel here!

🌱Your feedback helps us improve and grow together!

Stay up to date & join the conversation!

🍃 General Questions

What is the Lilypad Network?

Lilypad is developing a serverless, distributed compute network that enables internet-scale data processing for AI, ML & other arbitrary computation from blockchains, while unleashing idle processing power & unlocking a new marketplace for compute.

Lilypad provides decentralized AI computational services. By extending unrestricted, global access to computational power, Lilypad strategically collaborates with decentralized infrastructure networks, such as Filecoin, to formulate a transparent, efficient, and accessible computational ecosystem.

Perform off-chain decentralized compute over data, with on-chain guarantees, and to call this functionality directly from a smart contract, CLI and an easy to use abstraction layer, opens the door to a multitude of possible applications.

Lilypad Whitepaper

Lilypad will release a Whitepaper by November 2024.

Roadmap

Lilypad Network is currently in Testnet. The team is currently ironing out some remaining known issues and working on a fair model for our Incentivized Testnet.

Find the full Lilypad Network roadmap on our website!

Wait, didn’t Lilypad used to rely on determinism and optimistic reproducibility for verifiable compute?

Previously, Lilypad required deterministic jobs on the network and used optimistic reproducibility to randomly re-run jobs to ensure trust, however this method has been deprecated due to:

1. the limitation the determinism requirement placed on what jobs were available to run on the network

2. the inability to realistically be able to determine a “deterministic” job to be deterministic easily

Has Lilypad raised VC money?

Yes, Lilypad closed our seed round of funding in March 2024.

🌐 Incentivized Testnet Questions

When will the Incentivized Testnet launch?

The Lilypad Incentivized testnet launched in mid June 2024. Stay tuned on the Lilypad Discord for more info!

How do LilyBit_ rewards work?

Lilybit_ rewards will be awarded to nodes for time on the network (up to a 4x multiplier) and compute power brought to the network. Rewards will be redeemable, for the Lilypad ERC20 Utility Token at Mainnet Launch, with between 5% and 10% of the total token supply (depending on IncentiveNet demand and tokenomics finalization) being allocated to this phase of the Lilypad Network.

Resource Providers (RP) can track their Lilybit_ earnings with the RP Leaderboard.

Who can earn LilyBit_ rewards?

Phase 1 of the Incentivized Testnet is focused on rewarding nodes on the network, referred to as Resource Providers. The earlier a provider joins the network, the more Lilybits_ will be available.

Phases 2 and onward will provide rewards for Lilypad modules created as well as developer tools/apps (in addition to rewarding nodes).

How do I check my Lilybit_ rewards?

You can check your rewards by pasting your nodes wallet address into the following inteerfaces:

How does Lilypad use blockchain, and why do I need both ETH and Lilypad tokens to run a job?

On the Lilypad network, The blockchain is used for

• payment rails

• storing the deals transparently (on-chain guarantees about the compute)

• storing any disputes & results

Lilypad Tokens are used to transact on the Lilypad network. They are used as payment by those who want to run jobs (to the resource providers who run them), and as collateral by resource providers

You need ETH tokens to pay for the gas fees for smart contracts the facilitate transactions, and for records of transactions and disputes that are posted to the Ethereum blockchain.

• Grafana dashboard

• Lilypad leaderboard

• Open Source contributor dashboard

⚙️ Hardware Provider Questions

What do I need to do before I can run a Lilypad node?

The required steps before running a Lilypad node include adding the node, adding the Lilypad network information, obtaining tokens and installing the required software.

Refer to the Running a Node documentation and select your preferred platform (Linux or Docker) for a detailed guide on the prerequisites.

What are the hardware requirements to run a Lilypad node?

The minimum hardware requirements to run a Lilypad node are:

• Processor: Quad-core x64 Processor

• RAM: 16GB (see additional details below)

• Internet: 250Mbps Download Speed

• GPU: NVIDIA GPU with a minimum of 8GB VRAM (see additional details below)

For more information, please visit Hardware Requirements.

What are the updates required for maintaining my node software?

Resource providers are expected to have the latest Lilypad versions installed on their machines.

The instructions can be found in our installation documentation (select the Resource Provider tab).

How can I check the status of my Lilypad node?

To check if the RP is running use the following command: sudo systemctl status lilypad-resource-provider.

This will give a live output from the Lilypad node. The logs will show the node running and accepting jobs on the network. To get more information from your node, run the following: sudo journalctl -u lilypad-resource-provider.service -f.

Find more information, select your preferred platform below and check out the docs:

• Linux

• Docker

👩‍💻 Developer Questions

What is a Lilypad Module?

To run a ML model like Stable Diffusion on Lilypad, the model must be setup as a Lilypad Module (see instructions below). Once setup, modules are run with the Lilypad CLI.

How to run a ML job on Lilypad

To run a ML job on Lilypad (Stable Diffusion, Stable Diffusion Video, and more) using the Lilypad CLI, follow the CLI instructions to get started (select the CLI Users tab).

To build an application with Lilypad compute and modules on the backend, check out this guide.

How to add a ML model to run on Lilypad

A Lilypad module is a Git repository that can be used to perform various tasks using predefined templates and inputs. This "build a job module" guide will walk you through the process of creating a Lilypad module, including defining a JSON template, handling inputs, ensuring determinism, and other best practices.

How to run a Lilypad node

Lilypad is an open network that allows anyone to contribute GPU computing capacity. Find instructions for running a node by selecting your preferred platform below:

• Linux

• Docker

Node hardware specs

• Linux (latest Ubuntu LTS recommended)

• Nvidia GPU

• Nvidia drivers

• Docker

• Nvidia docker drivers

For more information on the requirements to run a Lilypad node, please refer to the hardware requirements documentation.

📖 Token Questions

Lilypad Tokenomics

Tokenomics and research papers are currently being developed and expected by Q4 2024.

Expected TGE

Although the launch date is not finalized, the launch of Lilypad Mainnet and the TGE for LP tokens is scheduled for Q2 2025.

Can't find the answer you were looking for? Join the Lilypad Discord server for live support! 🪷

This page is a dynamic work in progress! We're currently working on some better diagrams!

Lilypad Technical Architecture

Based on Academic Research

The architecture of Lilypad is inspired by the research paper titled "Mechanisms for Outsourcing Computation via a Decentralized Market." The paper introduces MODiCuM, a decentralized system that allows for computational outsourcing in an open market. Just like MODiCuM, Lilypad aims to create an open market of computational resources by introducing various decentralized services like solver, resource provider, job creator, mediator, and directory services. MODiCuM's unique approach to deterring misbehavior in a decentralized environment through dedicated mediators and enforceable fines has influenced Lilypad's own design, particularly in the areas of dispute resolution and system integrity.

For an in-depth understanding, you can read the paper here.

Abstract of the paper: Mechanisms for Outsourcing Computation via a Decentralized Market As the number of personal computing and IoT devices grows rapidly, so does the amount of computational power that is available at the edge. Since many of these devices are often idle, there is a vast amount of computational power that is currently untapped, and which could be used for outsourcing computation. Existing solutions for harnessing this power, such as volunteer computing (e.g., BOINC), are centralized platforms in which a single organization or company can control participation and pricing. By contrast, an open market of computational resources, where resource owners and resource users trade directly with each other, could lead to greater participation and more competitive pricing. To provide an open market, we introduce MODiCuM, a decentralized system for outsourcing computation. MODiCuM deters participants from misbehaving-which is a key problem in decentralized systems-by resolving disputes via dedicated mediators and by imposing enforceable fines. However, unlike other decentralized outsourcing solutions, MODiCuM minimizes computational overhead since it does not require global trust in mediation results. We provide analytical results proving that MODiCuM can deter misbehavior, and we evaluate the overhead of MODiCuM using experimental results based on an implementation of our platform.

Both Resource Proiders (GPU compute nodes) and those looking to run jobs on the Lilypad network need to set up a Metamask account in order to run jobs on Lilypad. The public key of your wallet address is how you are identified on the network, and is how you can look up the transactions you make on the Arbitrum Etherscan blockchain explorer. You need an account for running jobs, and a separate account for each GPU you want to set up on the network.

The wallet you use for your account must have both ETH (to run smart contracts on Ethereum) and Lilypad (LP) tokens in order to pay for jobs (or recieve funds for jobs) on the network.

Create a MetaMask wallet

End users of Lilypad can decide which crypto wallet they would like to use. In this guide, we advise using a MetaMask crypto wallet.

• Install MetaMask Extension

Use the Arbitrum Sepolia Testnet in MetaMask.

The Lilypad Testnet (IncentiveNet) is currently running on the Arbitrum L2 network built on Ethereum.

In order to change to the Arbitrum network in the wallet, open MetaMask and click the network button in the top left of the menu bar:

Then , select "Add network":

Next, select "Add a network manually":

Input all the required Arbitrum Sepolia Testnet network info, and then "Save":

Import the Testnet LP token

The wallet is now setup and will display an ETH (Arbitrum Sepolia) token balance. In order to also display the LP token balance, the LP token will need to be imported.

Select "Import tokens" at the bottom of the wallet:

Select "Custom token" and add the Lilypad token contract address and token symbol. Then "Save".

• Token contract address: 0x0352485f8a3cB6d305875FaC0C40ef01e0C06535

• Token symbol: LP

You should now see both ETH and LP listed in the wallet (initial ETH and LP balances will be 0).

Now you're ready to fund the wallet with testnet LP and ETH tokens!

Before you run a Lilypad job, make sure you have Lilypad CLI installed and have set a WEB3_PRIVATE_KEY env variable in your environment.

Run Cowsay

Run the command:

lilypad run cowsay:v0.0.4 -i Message="moo"

Wait for the compute to take place and for the results to be published:

View your results:

cat /tmp/lilypad/data/downloaded-files/QmXzVjk6Vm6fvZ9kn8yuaNbb1ioGsq3wiogGXX5w3xmqRQ/stdout

tldr: To obtain funds, first ensure the wallet is connected to the Arbitrum Sepolia network. then, collect LP and ETH tokens from these faucets:

• Lilypad Testnet tokens (LP)

• Arbitrum Sepolia ETH (3rd party faucet list)

Faucet Guide

Get Testnet LP tokens

Find out why you need tokens in the FAQs

Follow these steps to successfully claim your Testnet LP tokens:

1. Navigate to the Lilypad Testnet faucet.

2. Authenticate with Discord.

3. Copy your MetaMask wallet address into the input

4. Click "Request".

LP Testnet tokens will be sent to this wallet address.

Get Arbitrum Sepolia Testnet ETH

Get Arbitrum Sepolia ETH from this list of third party faucets. Each faucet is designed differently, so follow the instructions provided.

View tokens

With a balance of both LP and ETH, you're ready to run jobs with the Lilypad CLI!

is developing a serverless, distributed compute network that enables internet-scale data processing, AI, ML & other arbitrary computation, while unleashing idle processing power & unlocking a new marketplace for compute.

You can use Lilypad to run AI workload models including Stable Diffusion and Stable Diffusion Video, or you can add your own module to run on the Lilypad Network. Using Lilypads distributed compute nodes, you can build and run your own containerized workloads that require high-performance computing.

Overview

Perform off-chain decentralized compute over data, with on-chain guarantees. Call this functionality directly from a smart contract, CLI, and an easy to use abstraction layer.

The network is actively laying groundwork for multi-chain integration and the deployment of an incentivized testnet.

Lilypad has evolved from earlier versions (v0, v1 & v2), where the network served as a proof of concept for verifiable, decentralized compute directly from smart contracts. These earlier iterations established the groundwork for what is now a robust, scalable platform with expanded features and multichain support.

Bacalhau has been integral to Lilypad since its early versions (v0 and v1), serving as the backbone for verifiable off-chain compute. In these early iterations, Bacalhau was used for Proof of Concept projects, helping users execute decentralized compute jobs from smart contracts.

Objective and problem statement

Lilypad aims to mitigate the challenges predominantly associated with the accessibility of high-performance computational hardware. At present, numerous barriers impede developers and organizations from smoothly integrating projects that require high-performance computing, such as AI technologies, into their applications.

Unlike conventional centralized systems, where access to powerful compute hardware is restricted and costly, Lilypad endeavors to democratize this access. Through its verifiable, trustless, and decentralized computational network, Lilypad extends unrestricted, global access to computational power. By leveraging decentralized infrastructure networks such as Filecoin, Lilypad is strategically positioned to enhance the efficiency, transparency, and accessibility of high-performance computing hardware.

Applications

Perform off-chain decentralized compute over data, with on-chain guarantees, and to call this functionality directly from a smart contract, CLI, API and an easy to use abstraction layer, opens the door to a multitude of possible applications including:

• Inference AI jobs

• ML training jobs

• Invoking & supporting generic ZK computations

• Cross-chain interoperability complement to bridge protocols

• Utilising inbuilt storage on IPFS

• Federated Learning consensus (with Bacalhau insulated jobs)

• IOT & Sensor Data integrations

• Providing a platform for Digital twins

• Supply chain tracking & analysis

• ETL & data preparation jobs

Key features

Some of the key features of Lilypad include:

1. Verifiable Serverless Decentralized Compute Network: Lilypad is a decentralized compute network that aims to provide global, permissionless access to compute power. The Network orchestrates off-chain compute (a global GPU marketplace) and uses on-chain verification (Arbitrum L2 on Ethereum) to provide guarantees of compute success.

3. Open Compute Network: Lilypad is an open compute network allowing users to access and run AI models/other programs in a serverless manner. Module creators and general users can access a curated set of modules or can easily create their own Lilypad module to run an AI model/other program on the network.

6. Decentralization of Mediators: The team also aims to decentralize the mediators in the network. This means that the decision-making process and governance of the network will be distributed among multiple participants, ensuring a more decentralized and resilient system.

What is the Bacalhau Project?

Bacalhau is a peer to peer computation network enabling compute over data jobs like GPU-enabled AI, ML, analytics, data engineering, data science, de-sci and more. With the open-source Bacalhau Project, you can streamline your existing workflows without rewriting by running Docker containers and WebAssembly (WASM) images as tasks. This architecture is also referred to as Compute Over Data (or CoD).

Roadmap

Lilypad Linktree - Learn more and join the community!

Resources

Join the Community & Chat with Us

Prerequisites

• Linux (Ubuntu 22.04 LTS)

• Nvidia GPU

• (Ubuntu install)

• Nvidia Docker drivers

Network information and Testnet tokens

The testnet has a base currency of ETH, as well as a utility token called LP. Both are used for running nodes. To add a node to the testnet, follow these steps:

Metamask Configuration

We recommend using MetaMask with custom settings to make things easier. Once you have it installed and setup, here are the settings you need to use:

Fund your wallet with ETH and LP

The faucet will give you both ETH (to pay for gas) and LP (to stake and pay for jobs).

Setup Arbitrum RPC (Optional)

The Lilypad Network uses the Arbitrum Sepolia Testnet to settle compute transactions. When a transaction is ready to be saved on-chain, Lilypad cycles through a list of public Arbitrum Sepolia RPC endpoints using the endpoint that settles first to save the compute transaction.

Usage

You have two options to start the Lilypad setup: using Docker Compose or directly pulling the image. Both methods will run the containers in the background, allowing you to continue using your terminal while the setup operates.

Docker Compose Setup

1. Export Your Private Key

Before starting, export your private key from MetaMask. Follow the official MetaMask guide for instructions on safely exporting your private key.

2. Download the Docker Compose Configuration

Use curl to download the docker-compose.yml file from the Lilypad GitHub repository.

3. Check for Existing Containers

You can check if these containers are running with:

If they are running, stop them with:

If there are still conflicts when trying to running with the docker-compose file, remove the containers:

4. Start the Resource Provider

Start the Lilypad containers using Docker Compose:

To include a custom RPC URL:

5. Monitor Your Node

Use the following command to check the status of the Lilypad Resource provider.

Use the following command to view the containers running after starting Docker Compose.

View Lilybit_ rewards

To view your Lilybit_ rewards, visit one of the following dashboards and paste your node's public address into the input:

Troubleshooting

Here are some common troubleshooting techniques when it comes to your resource provider using Docker:

Checking Docker Runtime To verify your Docker runtime configuration: sudo docker info | grep Runtimes

You should see the NVIDIA runtime listed. If you only see: Runtimes: io.containerd.runc.v2 runc you will need to configure the NVIDIA runtime.

Configuring NVIDIA Runtime If the NVIDIA runtime is not showing up or you're experiencing issues, try the following:

1. Configure the NVIDIA Container Toolkit runtime: sudo nvidia-ctk runtime configure --runtime=docker 2. Restart the Docker service: sudo systemctl restart docker

Overview of Docker setup For a comprehensive overview of your Docker setup, use: docker info. This command provides detailed information about your Docker daemon configuration.

This guide walks through the steps of setting up a personal RPC endpoint for Arbitrum Sepolia using .

Setup Infura account

on Infura and choose your plan based on how many APIs you need.

Select the “free” tier as the compute units provided should be sufficient to run a Lilypad RP.

Setup RPC endpoint for Arbitrum Sepolia

In the Infura dashboard, a new API key will usually generate automatically. If not, select "Create New API Key". Navigate to "Configure" to setup the API key.

Scroll down the list to the Arbitrum network and ensure the Sepolia testnet box is checked, then save changes.

In the API key dashboard, select "Active Endpoints" and navigate to "WebSockets".

Scroll down the page to find the Arbitrum Sepolia URL. The RPC endpoint for Arbitrum Sepolia is ready to be used with the Lilypad Resource Provider:

Use the new RPC endpoint

Lilypad RPs can use a personal RPC endpoint with a few simple steps. Only Web-socket (WSS) connections are supported.

Docker users

Stop the existing Lilypad Resource Provider (RP) before setting up the new RPC.

Locate the Lilypad RP Docker container using:

Stop the container using the PID:

Use this command to start the lilypad-resource-provider.service with the new RPC:

Check the status of the container:

Ubuntu users

Stop the existing Lilypad RP (if the node is not running, disregard this first step):

Update lilypad-resource-provider.service with the new RPC:

Add following line to [Service] section:

Reboot the node:

The Lilypad Network uses the Arbitrum Sepolia Testnet to settle compute transactions.

This personal RPC endpoint allows Resource Providers (RP) to avoid reliability issues with the RPC endpoints used by Lilypad ensuring rewards can be earned and jobs can be run consistently. RPs running a personal RPC endpoint contribute to the fault tolerance and decentralization of the Lilypad Network! Read more in the Alchemy Arbitrum .

Setup RPC video guide

This page overviews the hardware requirements to operate a Lilypad Network node. It's important to note that these requirements continuously evolve as the network grows. If you have questions or suggestions, please join our or open a pull request on the .

Minimum Hardware Requirements

• Processor: Quad-core x64 Processor or better

• RAM: 32GB (see additional details below)

• Internet: Internet: 250Mbps download, 100Mbps upload (minimum)

• GPU: NVIDIA GPU with a minimum of 8GB VRAM (see additional details below)

GPU Requirements

Each model operating on Lilypad has specific VRAM (Video Random Access Memory) requirements directly related to the model's complexity and computational demands. For running a Lilypad Resource Provider (RP) with multiple GPUs, a guide using Proxmox can be found .

• Base Requirement: The simplest model on Lilypad requires a GPU with at least 8GB of VRAM. This is the minimum required to participate in computational tasks on the Lilypad network.

Model-Specific VRAM Requirements

The capability of your GPU to manage multiple or more complex Lilypad jobs is enhanced by the amount of VRAM available:

• Standard Models (, ): Requires 8GB of VRAM.

• Advanced Models (): Requires 14GB of VRAM.

Hardware Compatibility

• GPUs with 8GB of VRAM are limited to running models like SDXL, which fit within this specification. Larger GPUs with higher VRAM are required for more demanding models like SDV, which needs at least 14GB of VRAM.

For example:

RAM Requirements

Lilypad uses the Resource Provider's GPU to load models, initially requiring the temporary storage of the data in the system's RAM. In a production environment with RP Nodes, it is important to have enough RAM to support the model and the underlying system's operational processes.

Minimum RAM Requirements:

• Base Requirement: A minimum of 16GB of RAM is required, with at least 8GB dedicated to the model and another 8GB allocated for system operations and Lilypad processes.

Additional Considerations

• Wallets for each GPU: You need a separate account for each GPU you want to set up on the network. The wallet you use for your account must have both ETH (to run smart contracts on Ethereum) and Lilypad (LP) tokens in order to recieve funds for jobs) on the network.

• Larger Models: Jobs involving more substantial models will demand additional RAM. It's important to note that adequate VRAM alone is insufficient; your system must also have enough RAM to load the model into the GPU successfully. Without sufficient system RAM, the model cannot be loaded into the GPU, regardless of available VRAM.

GPUs contributing to the Lilypad Network can earn Lilybit_ credits daily, which will be redeemable for Lilypad (LP) mainnet tokens at the Token Generation Event (TGE), expected in Q1 of 2025. To provide compute power and participate in Lilypad's decentralized compute network, Resource Providers (RPs) can accumulate these credits based on their contributions to the network.

Read the in depth breakdown of the Lilybit_ rewards system .

tldr:

• Nodes must be online for a minimum of 4 hours a day continuously (verified with POW efforts) to earn that day’s Lilybit_ rewards.

• A 4x multiplier on this base daily rate is available for nodes that are online and taking jobs over the full day (This is calculated with 1.3195^(number of 4 hour windows GPU is online)).

• The power of the compute being contributed also provides a point multiplier, with the hashrate of a node (determined by the required POW algorithm) determining the multiplier given to a node’s power. 10 points are rewarded for every MHash/sec provided to the network.

• In order to incentivize long term participation in the network, a node will be slashed 10% of their total number of points (earned by providing compute power) each day that node isn’t online for at least 4 hours continuously.

• A grace period for RP downtime is now included in the slashing mechanism (Oct-01-2024). RPs will earn 2 days of a “grace period” after every 30 days of continuous service provided. These 2 days will be applied to 2 subsequent down days recorded by the RP allowing the RP to avoid slashing for these 2 days. Grace Period days do not accumulate to more than 2 days ever. Once used the 30 day count to obtain the 2 days restarts. Find more information on slashing in the .

Rewards Leaderboard

The provides an up-to-date view of rewards earned by each wallet ID and is updated once a day to reflect additional credits earned. RPs can also track the status of their compute contributions with the node status dashboard.

Track the status of a Resource Provider with the .

This guide walks through the steps of setting up a personal RPC endpoint for Arbitrum Sepolia using .

Setup Alchemy account

and login to the Alchemy dashboard.

Select the “free” tier as the compute units provided should be sufficient to run a Lilypad RP. The free service provides 300 million compute units per month.

Select “skip bonus” or input a credit card with billing info (the card will not be charged unless the compute credits in the free tier are used).

Setup RPC endpoint for Arbitrum Sepolia

In the “Overview” section of the Alchemy dashboard, navigate to “My app” and select “Endpoints”. If an app was not created upon login, create a new one by selecting "Create new app".

By selecting “Endpoints”, the “Networks” tab will open providing an option to configure the Arbitrum API.

• Select “Sepolia”

• Select “Websockets”

The RPC endpoint for Arbitrum Sepolia is ready to be used with the Lilypad Resource Provider:

Metrics for the RPC can be viewed in the “Metrics” tab.

Use the new RPC endpoint

Lilypad RPs can use a personal RPC endpoint with a few simple steps. Only Web-socket (WSS) connections are supported.

Docker users

Stop the existing Lilypad Resource Provider (RP) before setting up the new RPC.

Locate the Lilypad RP Docker container using:

Stop the container using the PID:

Use this command to start the lilypad-resource-provider.service with the new RPC:

Check the status of the container:

Ubuntu users

Stop the existing Lilypad RP (if the node is not running, disregard this first step):

Update lilypad-resource-provider.service with the new RPC:

Add following line to [Service] section:

Reboot the node:

Lilypad enables you to run compute jobs on the network of the network of computers on Lilypad Tetsnet.

The cloud is just somebody else's computer...

Overview

This guide will take you through:

A Lilypad module is a Git repository that allows you to perform various tasks using predefined templates and inputs. This guide will walk you through creating a Lilypad module, including defining a JSON template, handling inputs, and following best practices.

Module Structure

Start by creating a Git repository for your Lilypad module. The module's versions will be represented as Git tags. Below is the basic structure of a Lilypad Module.

Prepare Your Model

• Handle all dependencies

• Implement input/output through environment variables

• Write outputs to /outputs directory

1. Create Run Script (run_model.py for example) that will be used in conjunction with Docker

2. Create a Dockerfile that functions with your run script

3. Build and Publish Container(example below uses Dockerhub for storage)

4. Create a lilypad_module.json.tmpl Template

Environment Variables

Format in template:

Usage in CLI:

lilypad run repo:tag -i variable=value

Formatting your module run command

During development, you will need to use the Git hash to test your module. This allows you to verify that your module functions correctly and produces the expected results.

Below is a working lilypad module run cmd for reference. (you can use this to run a lilypad job within the lilypad CLI):

Examples

Here are some example Lilypad modules for reference:

Deprecated examples:

These examples can help you understand how to structure your Lilypad modules and follow best practices.

Conclusion

In this guide, we've covered the essential steps to create a Lilypad module, including defining a JSON template, handling inputs, and testing your module. By following these best practices, you can build reliable and reusable modules for Lilypad.

For more information and additional examples, refer to the official Lilypad documentation and the Cowsay example module.

Prerequisites

• Linux (Ubuntu 22.04 LTS)

• Nvidia GPU

• (Ubuntu install)

• Nvidia Docker drivers

Network information and testnet tokens

The testnet has a base currency of ETH, as well as a utility token called LP. Both are used for running nodes. To add a node to the testnet, follow these steps:

Metamask

We recommend using MetaMask with custom settings to make things easier. Once you have it installed and setup, here are the settings you need to use:

Fund your wallet with ETH and LP

The faucet will give you both ETH (to pay for gas) and LP (to stake and pay for jobs).

Installation

To set up your environment for using Lilypad with GPU support, you need to install several key components. This guide will walk you through installing Docker, the Nvidia Container Toolkit, Bacalhau, and Lilypad. You'll also configure systemd to manage these services efficiently.

Install Docker

Docker is a platform that allows you to automate the deployment of applications inside lightweight, portable containers.

To install Docker Engine, follow the steps specific to your operating system from the official Docker documentation:

Install Nvidia Container Toolkit

Configure the container runtime by using the nvidia-ctk command:

The nvidia-ctk command modifies the /etc/docker/daemon.json file on the host. The file is updated so that Docker can use the NVIDIA Container Runtime.

Restart the Docker daemon:

Install ipfs

Run a local ipfs node on the Lilypad RP.

Install Bacalhau

Bacalhau is a peer-to-peer network of nodes that enables decentralized communication between computers. The network consists of two types of nodes, which can communicate with each other.

To install Bacalhau, run the following in a new terminal window (run each command one by one):

To check your Bacalhau version use:

The expected output is:

Install Lilypad

The installation process for the Lilypad CLI involves several automated steps to configure it for your specific system. Initially, the setup script identifies your computer's architecture and operating system to ensure compatibility. It will then download the latest production build of the Lilypad CLI directly from the official GitHub repository using curl and wget.

Once the CLI tool is downloaded, the script sets the necessary permissions to make the executable file runnable. It then moves the executable to a standard location in your system's path to allow it to be run from any terminal window.

Via official released binaries

To verify the installation, run lilypad in the terminal to display the version and a list of available commands, indicating that Lilypad CLI is ready to use.

Write env file

You will need to create an environment directory for your node and add an environment file that contains your node's private key.

To do this, run the following in your terminal:

Next, add your node's private key into /app/lilypad/resource-provider-gpu.env:

This is the key where you will get paid in LP tokens for jobs run on the network.

Setup Arbitrum RPC

The Lilypad Network uses the Arbitrum Sepolia Testnet to settle compute transactions. When a transaction is ready to be saved on-chain, Lilypad cycles through a list of public Arbitrum Sepolia RPC endpoints using the endpoint that settles first to save the compute transaction.

Install systemd unit for Bacalhau

systemd is a system and service manager for Linux operating systems. systemd operates as a central point of control for various aspects of system management, offering features like parallelization of service startup, dependency-based service management, process supervision, and more.

To install systemd, open /etc/systemd/system/bacalhau.service in your preferred editor:

Install systemd unit for GPU provider

Open /etc/systemd/system/lilypad-resource-provider.service in your preferred editor.

Reload systemd's units/daemons (you will need to do this again if you ever change the systemd unit files that we wrote, above)

Start Lilypad node

Start systemd units:

Now that your services have been installed and enabled, check the status of Bacalhau to ensure it is running correctly on your node:

View node status

To check if the node is running use the following command:

This will give a live output from the Lilypad node. The logs will show the node running and accepting jobs on the network.

Run the following command to get more status info from your node:

To restart your resource provider run:

Support for Lilypad RPs

• Without a discussion opened, our team will not be able to support the problem.

• Description (including Lilypad version running on your node)

• Hardware Info (including Linux/Windows version)

• Related blockchain/ETH addresses of transaction hashes

• Output Logs - sudo systemctl status lilypad-resource-provider

• Related links/urls

• Screenshots

Update Lilypad version

Please note that using sudo rm -rf is very powerful and can be dangerous if not used carefully.

1. If the Lilypad RP is running, stop the system (if the node is not running, disregard this first step):

1. Remove the Lilypad executable by running:

2. Start your resource provider by running:

Disconnecting a node

To disconnect your node from Lilypad you will need to do a few things to completely offboard.

First, stop the node:

Next, you must remove the .service files related to Lilypad and Bacalhau. These files are typically stored in /etc/systemd/system/. To remove them, run the following command:

Next we notify the systemd manager to reload its configuration by running:

Then, remove the environment file for the Lilypad resource provider. This file is usually stored in /app/lilypad/. To remove it, run:

Finally, if you followed the installation instructions from the Lilypad documentation and moved the executable to /usr/local/bin/lilypad, it can be removed from there. If the executable is stored in a different directory on your machine, navigate to that directory and remove it from there. To remove the executable, run:

To remove Bacalhau, run:

View Lilybit_ rewards

To view your Lilybit_ rewards, visit one of the following dashboards and paste your node's public address into the input:

Security

If you want to allowlist only certain modules (e.g. Stable Diffusion modules), to control exactly what code runs on specific nodes (which can be audited to ensure that they are secure and will have no negative impact on the nodes), set an environment variable OFFER_MODULES in the GPU provider to a comma separated list of module names, e.g. sdxl:v0.9-lilypad1,stable-diffusion:v0.0.1.

Run a node video guide

Overview

Based on ComfyUI, the SDV Pipeline modules for Lilypad allow you generate videos from text prompts on Lilypad using Stable Diffusion Video and related models.

The SDV Pipeline modules are designed to take your text prompt, generate a still frame using SDXL, then use that as the input to the SDV model, producing an APNG (animated PNG), WebP video, and an MP4 video all in one go.

Getting Started

Prerequisites

Before running sdv, make sure you have the on your machine and your private key environment variable is set. This is necessary for operations within the Lilypad network.

Running SDV v1.0 or 1.1

To run SDV v1.0 or 1.1 Pipeline in Lilypad, you can use the following commands:

SDV 1.0

SDV 1.1

SDV Output

To view the results in a local directory, navigate to the local folder.

To view the results on IPFS, navigate to the IPFS CID result output.

As Lilypad modules are currently deterministic, running this command with the same text prompt will produce the same image, since the same seed is also used (the default seed is 0).

Tuning an output

If you wish to specify more than one tunable, such as the number of steps, simply add more -i flags. For example, to change or improve the quality of the image generated add "Steps=x" with x = (min: 5. Max: 200):

Options and tunables

The following tunables are available. All of them are optional, and have default settings that will be used if you do not provide them.

Overview

Based on Ollama, the Ollama Pipeline modules for Lilypad allow you generate text on Lilypad using various models.

Available models

• Llama3

Llama3

Llama3 is a machine learning model used for natural language processing. It is based on a transformer architecture, which enables it to handle tasks like text generation, summarization, translation, and more. When integrated with Lilypad, it leverages the platform's capabilities to provide efficient text processing for various applications.

Prerequisites

Before running the Ollama Pipeline, make sure you have the on your machine and your private key environment variable is set. This is necessary for operations within the Lilypad network.

Usage

The Ollama Pipeline in Lilypad can be run using the Lilypad CLI or Docker. Below are the instructions for both of those options.

Lilypad

To run Ollama Pipeline using the Lilypad CLI, you can use the following command:

Docker

To run this module in Docker, you can use the following commands:

Links

cowsay is a simple, text-based program originally written for Unix-like operating systems that generates ASCII pictures of a cow with a speech bubble containing a specified message.

This module was created as a "Hello World" for the Lilypad Network!

Getting Started

Prerequisites

Before running cowsay, make sure you have the on your machine and your private key environment variable is set. This is necessary for operations within the Lilypad network.

Run cowsay

Once you've installed the CLI, run the cowsay command:

Output

To view the results in a local directory, navigate to the local folder.

Here, you can view the stdout and stderr as well as the outputs folder for the run:

Please be patient when waiting for the IPFS result. It can take some time to propogate and doesn't always work immediately.

Prior Verification-via-Replication Protocols

Before explaining our approach, we give a short overview of three prior approaches to verification-via-replication distributed computing protocols: Truebit, Modicum, and Smart Contract Counter-Collusion. We will outline potential improvements, and how our work builds on top of, and differs from, prior work.

Truebit is a protocol for outsourcing computation from blockchains, built using smart contracts on top of Ethereum. The original potential use cases were trustless mining pools, trustless bridges, scaling transaction throughput, and, scalable “on-chain” storage. Since its goal is to scale on-chain computation, it aims for global consensus: "Since there exist no trusted parties on Ethereum’s network, by symmetry we must allow any party to be hired to solve any computational task, and similarly anyone should be able to challenge a Solver’s outcome of a computational task. The latter requirement ensures that TrueBit operates by unanimous consensus." (emphasis added)

The components of Truebit are Miners, Task Givers, Solvers, Verifiers, and Judges. In order to incentivize checking results, random errors are forced into computations, with jackpots awarded to those who find them. These jackpots are funded by taxes on computations.

The verification game consists of a series of rounds, where in each round, a smaller and smaller subset of the computation is checked. Eventually, only one instruction is used to determine whether a Solver or Verifier is correct:

The authors claim that Sybil attacks are mitigated by pairwise Sybil-resistance between the parties of Task Givers, Solvers, and Verifiers, with Judges and Referees, whose roles are played by Miners, assumed to function as intended. Likewise, they claim that attacks to get bogus solutions on-chain by scaring off Verifiers are mitigated by the economics of deposits, taxes, and jackpot rewards. Additionally, a cartel of Solvers who absorb losses until they find a task with a forced error, upon which time they will receive the jackpot, will lose money in the long-term, since the expected Solver deposit per task is higher than the expected jackpot per task. Addressing another attack, the authors claim that an attack involving a flood of Challengers who try to take as much of the jackpot reward resulting from a forced error as possible is mitigated by having the total jackpot reward decrease as the number of Challengers increases.

Challenges

• Does not scale to large/complicated/arbitrary computations

• No formal theorems or proofs, no simulations, many plausible but unsubstantiated claims, especially regarding collusion

• Everything is done on-chain

• This model does not work well with two-sided marketplaces, because

  • It aims for global consensus, where any node is allowed to do the computation, whereas in two-sided marketplaces, clients need to be able to choose which nodes they are paying to do the computation

  • Clients may have time restrictions on their computations, and cannot wait for cases where their computations were injected with forced errors

• No accounting for repeated games

Takeaways

• Taxes and jackpots are a valuable tool to create a global game that affects local outcomes

• Provides a list of potential client attacks

Job Creators are only allowed to submit deterministic jobs to the protocol. The Mediator exists to check for non-deterministic tasks submitted by the Job Creator (which can be used by the Job Creator as an attack vector to get free results), and fake results returned by the Resource Provider. The method for determining whether a job is deterministic or not is for the Mediator to run a job n times and check to see whether it receives different answers.

Modicum combines two separate ideas: checking the result from a Resource Provider to see if it is correct, and checking a result from a job submitted by a Job Creator to see if the job was deterministic or not. This implies that there is no capability for a client to simply check whether a result is correct or not, without the possibility of its collateral being slashed.

An alternative to trusting the Mediator (to act as a TTP) by having it run a job n times is having a consortium of n Mediators each run the task a single time. However, this adds the complication of achieving consensus in that consortium.

The issue of the Resource Provider and Mediator colluding to return a fake result is not addressed by this protocol. The authors allow for a Job Creator or Resource Provider to remove a Mediator from their list of trusted Mediators if they no longer trust it. However, that still leaves room to cheat at least once, and ideally this should be disincentivized from the outset.

There is also the issue of collateralization. The Modicum protocol, as well as a number of other protocols, assume (approximate) guesses as to the cost of jobs in advance, so that nodes can deposit the correct amount of collateral. However, doing so is fraught with complications; we provide an alternative approach in the Mechanisms to Explore section.

Challenges

• The Mediator is basically a trusted third party

• Client cannot simply check a result without being slashed, which is a consequence of the client attack model

• No accounting for repeated games

Takeaways

• Potential client attack, though one that can be mitigated by technical means

• The client has benefit of getting correct results, which needs to be accounted for in simulation

The authors determine that cryptographic methods for verifiable computation are too expensive for real-world scenarios. For that reason, they rely on verification-via-replication. The scenario is one in which a client simultaneously outsources computation to two clouds, where those two clouds deposit collateral into smart contracts in such a way to create a game between them, where the game incentivizes doing the computation honestly. The central challenge that the authors tackle is the issue of collusion - that is, what if the two clouds collude on an incorrect answer?

In contrast to Modicum, the client is assumed to be honest, and in contrast to Truebit, a trusted third part (TTP) is used to handle disputes.

Three Contracts

The authors use a series of three contracts to counter collusion.

The first game is an induced Prisoner's Dilemma - to avoid the two clouds colluding, one cloud can be rewarded the other cloud's deposit (minus payment to the TTP for resolving the dispute) if the former returned the correct result and the latter did not. Thus, each cloud is better off giving the other cloud fake results while computing the correct result itself. This contract is called the Prisoner's contract. It is analogous to the equilibrium in the classic prisoner's dilemma being defection <> computing honest result and giving other node fake result if offered to collude.

However, the clouds can agree to collude via a smart contract as well. They could do this by both depositing another amount into a contract, where the leader of the collusion must add a bribe (less than its cost of computing) to the contract as well (disregarding the bribe, both clouds deposit the same amount of collateral). The deposit is such that the clouds have more of an incentive to follow the collusion strategy than to deviate from it. This contract is called the Colluder's contract.

In order to counteract the Colluder's contract, a Traitor's contract is used to avoid this scenario by incentivizing the clouds to report the Colluder's contract. The basic concept is that the traitor cloud indeed reports the agreed-upon collusion result to the client in order to avoid the punishment in the Colluder's contract, but also honestly computes and returns the result to the client in order to avoid the punishment of the Prisoner's contract. The client must also put down a deposit in the Traitor's contract. Only the first cloud to report the Colluder's contract gets rewarded. The signing and reporting of the contracts must happen in a particular order in order for this process to work.

The authors prove that these games individually and together lead to a sequential equilibrium (which is stronger than a Nash equilibrium), meaning that it is optimal not only in terms of the whole game, but at every information set (basically the set of options each player has at every turn).

Challenges

• A Colluder's contract can be signed on different chains (or even off-chain). In order to mitigate this, the Traitor's contracts would have to become cross-chain (which is a major technical challenge), not to mention the possibility of cryptographically secure contracts (e.g. MPC contracts) where one of the parties alone would not be able to prove the existence of this contract

• Relies on trusted third party to resolve disputes

• Every task is replicated (that is, two copies of each job are always computed)

• Assumes client is honest

• Assumes amount of collateral known beforehand

• No accounting for repeated games

  • It is well known that in the repeated Prisoner's dilemma, depending on the assumptions, cooperation becomes the equilibrium

Takeaways

• The contracts and the payoffs that they induce offer a valuable toolbox to think about the problem of collusion

• The contracts offer, in a restricted setting, an ironclad way (assuming the proofs are correct) of preventing collusion

Using the Lilypad Contracts

We have deloyed the LilypadOnChainJobCreator contract which you can use to trigger running jobs on the lilypad network from other smart contracts.

It works in tandem with the lilypad jobcreator on-chain which will watch the on-chain contract and manage jobs on behalf of contracts that submit them.

Creating a job

You will need to know the contract address for the on-chain job creator so we can submit transactions to it.

The production controller address is 0x8e136587e3e5266d5244f6aa896E5CAf8E969946 and you can ask it for the address of the on-chain job creator getJobCreatorAddress()

Running a job involves 2 phases:

• calling approve on the ERC-20 contract to allow the solver to spend your tokens

• trigger the job via the on chain job manager

Now we know the address of the on-chain job controller - we can ask it for 3 things:

• the address of the ERC-20 token contract - getTokenAddress()

• how much the required deposit it - getRequiredDeposit()

• the address of the solver that will handle running the job for us - getControllerAddress()

Knowing these 3 things means we can call the standard ERC-20 approve to allow the solver to spend our tokens on our behalf.

Now - we can call the runJob method of the on chain controller from another contract. This will cause the job-creator service to kick in and do the following things:

• check that funds have been approved for the solver

• transfer those funds to it's wallet

• run the job on lilypad

• call the submitResults method on the on-chain job creator

• the on-chain job creator will call the submitResults of the original calling contract

The following is an example on-chain smart contract:

// SPDX-License-Identifier: Apache-2.0
pragma solidity ^0.8.6;

import "@openzeppelin/contracts/access/Ownable.sol";
import "@openzeppelin/contracts-upgradeable/proxy/utils/Initializable.sol";
import "./ILilypadJobManager.sol";
import "./ILilypadJobClient.sol";

contract ExampleClient is Ownable, Initializable, ILilypadJobClient {

  address private jobManagerAddress;
  ILilypadJobManager private jobManagerContract;

  mapping(uint256 => string) private jobResults;

  event JobCreated(
    uint256 id,
    string message
  );

  event JobCompleted(
    uint256 id,
    string dealId,
    string dataId
  );

  function initialize(address _jobManagerAddress) public initializer {
    setJobManagerAddress(_jobManagerAddress);
  }

  function setJobManagerAddress(address _jobManagerAddress) public onlyOwner {
    require(_jobManagerAddress != address(0), "Job manager address");
    jobManagerAddress = _jobManagerAddress;
    jobManagerContract = ILilypadJobManager(jobManagerAddress);
  }

  function getJobResult(uint256 _jobID) public view returns (string memory) {
    return jobResults[_jobID];
  }

  function runCowsay(
    string memory message
  ) public {
    string[] memory inputs = new string[](1);
    inputs[0] = string(abi.encodePacked("Message=", message));
    uint256 id = jobManagerContract.runJob(
      "cowsay:v0.0.4",
      inputs,
      msg.sender
    );

    emit JobCreated(
      id,
      message
    );
  }

  function submitResults(
    uint256 id,
    string memory dealId,
    string memory dataId
  ) public override {
    jobResults[id] = dataId;
    emit JobCompleted(
      id,
      dealId,
      dataId
    );
  }
}

Here is an example of a script that brings all of this together:

import bluebird from 'bluebird'
import {
  connectToken,
  connectJobManager,
  connectExampleClient,
  getWallet,
  getAddress,
} from '../utils/web3'
import { ethers } from 'hardhat'

async function main() {
  // it's annoying to not be able to use argv but hardhat complains about it
  const message = process.env.MESSAGE || 'Hello World!'

  const token = await connectToken()
  const manager = await connectJobManager()
  const client = await connectExampleClient()

  const setRequiredDepositTx = await manager
    .connect(getWallet('solver'))
    .setRequiredDeposit(ethers.parseEther("2"))
  await setRequiredDepositTx.wait()

  const requiredDeposit = await manager.getRequiredDeposit()

  console.log(`requiredDeposit: ${Number(requiredDeposit)}`)

  const paytokensTx = await token
    .connect(getWallet('job_creator'))
    .approve(getAddress('solver'), requiredDeposit)
  await paytokensTx.wait()

  console.log(`tokens approved: ${paytokensTx.hash}`)

  const runjobTx = await client
    .connect(getWallet('job_creator'))
    .runCowsay(message)
  const receipt = await runjobTx.wait()
  if(!receipt) throw new Error(`no receipt`)

  console.log(`submitted job: ${runjobTx.hash}`)

  let jobID = 0

  receipt.logs.forEach(log => {
    const logs = client.interface.parseLog(log as any)
    if(!logs) return
    jobID = Number(logs.args[0])
  })

  console.log(`Job ID: ${jobID}`)
  console.log(`Waiting for job to be completed...`)

  let result = ''

  while(!result) {
    result = await client.getJobResult(jobID)
    if(!result) {
      await bluebird.delay(1000)
    }
  }

  console.log(`Job result: ${result}`)
}

main().catch((error) => {
  console.error(error);
  process.exitCode = 1;
});