These documents provide a reference for outdated Lilypad versions.

Learn more about:

Overview

Lilypad is a ‘bridge’ to enable computation jobs from smart contracts. The aim of Lilypad v0 was to create an integration for users to call Bacalhau jobs directly from their solidity smart contracts and hence enable interactions and innovations between on-chain and off-chain compute. Lilypad v0 is a proof of concept bridge which runs off the public (free to use) Bacalhau compute network. As such, the reliability of jobs on this network are not guaranteed.

If you have a need for reliable compute based on this infrastructure - get in touch with us.

A user contract implements the LilypadCaller interface and to call a job, they make a function call to the deployed LilypadEvents contract.

This contract emits an event which the Lilypad bridge daemon listens for and then forwards on to the Bacalhau network for processing.

Once the job is complete, the results are returned back to the originating user contract from the bridge code.

See more about how Bacalhau & Lilypad are related below

The Lilypad Events contract - used for triggering compute jobs on Bacalhau, is currently integrated to the following networks on the address specified:

Lilypad v0 Deployed Networks

\

Introduction

This example uses the Stable Diffusion Docker image found on the Bacalhau Docs.

For more info on how to create the Stable Diffusion Script and Docker Image see this tutorial or watch the video below.

Code

This example can be found in the Examples folder in the Lilypad Project Github.

To run this example, you can simply deploy it to any supported network and pass in the contact address to the constructor which corresponds to your network. See Deployed Network Details.

Job Results

In this example the Lilypad Stable Diffusion Job Results are returned as an IPFS v0 CID. You can use Brave browser or an IPFS gateway to see the results.

For example for a given return CID, type the following into your browser url

• In Brave: ipfs://[CID]

• In Other Browsers: www.w3s.link/ipfs/[CID] or www.ipfs.io/ipfs/[CID]

A folder of the Outputs will be shown:

You can also access the image directly by navigating to

• In Brave: ipfs://[CID]/outputs/image0.png

• In Other Browsers: www.w3s.link/ipfs/[CID] or www.ipfs.io/ipfs/[CID]/outputs/image0.png

Fraud Detection

Introduction

This example uses the Fraud Detection Docker image found on .

Run the Image with Bacalhau using the command bacalhau docker --network=http --domain=api.etherscan.com run hakymulla/hackfs:inference -- python predict.py --address "0x427aE6048C7d2DEd45a07Ea46F2873d0F9ddDb35"

Run Bacalhau get [Job ID] to the format of the spec needed to call runLilypadJob.

For more info on the dataset, how to create the Fraud Detection Script and Docker Image see .

Code

This example can be found in the Examples folder in the Lilypad-v0 Project Github.

To run this example, you can simply deploy it to any supported network and pass in the contact address to the constructor which corresponds to your network. See deployed-network-details.md.

Walkthrough

To Deploy on Filecoin Calibration Net, use the LilypadEvents Contract Address deployed on the testnet (0xdC7612fa94F098F1d7BB40E0f4F4db8fF0bC8820).

Call the IsFraudDetector function and add an Ethereum Address as input.

IsFraudDetector is a payable function that calls the the bridge runLilypadJob.

Job Results

In this example the Lilypad FraudDetection Job Results is returned as a StdOut. You can use the allResult function to see the results.

With Tableland

Let's build on the above example by Creating and Interacting with Tableland table using smart contracts.

During contract deployment, a tableland table is created. The bridge while calling the lilypadFulfilled also saves the results of inference to the table which can be used to monitor the model perfomance.

Storing a reference to the table ID and prefix within your contract is advisable since it enhances ease of reference in different methods, facilitating smoother execution.

Create a table

The TablelandDeployments contract helps set up the TablelandTables interface by returning ITablelandTables instantiated with the correct registry address, inferred by the chain being used. This is added to the constructor. The table columns includes an id, address and the result.

Write table data

A insert function is created to write to the above table

notice the onlyBridge modifier, only the bridge can write to the table and happens during the call back.

The above insert function is called in the lilypadFulfilled function

What do I need to know to make Bacalhau Jobs?

• Bacalhau is language-agnostic, and supports Docker or WASM workloads. As long as you can run your executable in a container, you can run it in Bacalhau.

• You need to supply a Bacalhau job spec. To create a job spec, you can:

  • Run a Bacalhau job successfully, and then get the job spec back using bacalhau describe <job_id> --format=json.

  • Generate a job spec without running anything, using bacalhau docker run --dry-run.

  • Writing a job spec by hand, by using our schema as a guide.

• What can I do with Bacalhau now? You can:

  • read from IPFS, Filecoin, or URLs

  • write into Estuary or IPFS

Bacalhau Specification in JSON format

Bacalhau operates by executing jobs within containers. This means it is able to run any arbitrary Docker jobs or WASM images

Here's an example JSON job specification for a Stable Diffusion job which runs in a Docker container, requires no verification and publishes to Estuary.

{
    "Engine": "docker", 
    "Verifier": "noop",
    "PublisherSpec": {"Type": "estuary"},
    "Docker": {
    "Image": "ghcr.io/bacalhau-project/examples/stable-diffusion-gpu:0.0.1",
    "Entrypoint": ["python", "main.py", "--o", "./outputs", "--p", "A User Prompt Goes here"]},
    "Resources": {"GPU": "1"},
    "Outputs": [{"Name": "outputs", "Path": "/outputs"}],
    "Deal": {"Concurrency": 1}
}

Here's an example of using this JSON specification in a solidity smart contract:

Note that since we need to be able to add the user prompt input to the middle of the spec (in the Docker entrypoint), it's been split into 2 parts.

string constant specStart = '{'
    '"Engine": "docker",'
    '"Verifier": "noop",'
    '"PublisherSpec": {"Type": "estuary"},'
    '"Docker": {'
    '"Image": "ghcr.io/bacalhau-project/examples/stable-diffusion-gpu:0.0.1",'
    '"Entrypoint": ["python", "main.py", "--o", "./outputs", "--p", "';

string constant specEnd =
    '"]},'
    '"Resources": {"GPU": "1"},'
    '"Outputs": [{"Name": "outputs", "Path": "/outputs"}],'
    '"Deal": {"Concurrency": 1}'
    '}';

//Example of use:
string memory spec = string.concat(specStart, _prompt, specEnd);

Any Example Jobs?

There is a full complement of example jobs you can leverage on the Bacalhau Docs Site

Try out

• YOLO

• OCR

• Video Editing

• and many, many more!

Introduction

This example illustrates how DeFi Kicks leverages Lilypad, Bacalhau, and Drand to implement an unbiased off-chain voting mechanism. The purpose of using these technologies is to ensure that voting happens in an unbiased, decentralized manner, thus promoting true democracy within the DefiKicks ecosystem.

The contracts and docker images have also been added as an example in Lilypad Github

For more info on DeFi Kicks and their use of Lilypad, see the DefiKicks Github

Overview

The DeFi Kicks project utilizes a three-step mechanism to ensure an unbiased voting system. Here's a summarized outline of how it operates:

1. Off-Chain Voting

The voting process is performed off-chain to maintain efficiency and privacy. Users cast their votes by encrypting them on the front end, signing the message for authenticity, and then sending them to be stored. Time-lock encryption is utilized, making the vote only decryptable after a predetermined amount of time, which ensures fairness and protects against any attempts to tamper with the votes.

2. Vote Resolution Request

Once the voting phase for a proposal has ended, anyone can request a vote resolution. This action requires a certain fee and can only be performed if the vote is in the ResolutionToRequest state. This fee covers the usage of Lilypad and Bacalhau for off-chain vote resolution.

3. Off-Chain Resolution in Bacalhau through Lilypad

Once a vote resolution request has been submitted and the necessary fee paid, Bacalhau, via Lilypad, takes care of the off-chain resolution. It processes the vote data, decrypting the votes when the predetermined time has elapsed, and generates the final voting results. This off-chain resolution ensures a high level of security and accuracy, providing an unbiased outcome for the voting process.

In summary, this mechanism leverages the strengths of off-chain voting and resolution in a decentralized environment, guaranteeing privacy, fairness, and tamper resistance in the voting process, which are all critical to maintaining the trust and integrity of the DeFi Kicks project.

Technical Details

1. Off-Chain Voting

Vote details are packaged into a message object and signed cryptographically. This message is then encrypted using a time-lock mechanism, which ensures decryption only after a certain time. The encrypted vote, along with the user's account details and signature, are stored as a vote object. Lastly, the new vote is stored and the system is notified.

The following code snippet shows how the off-chain voting mechanism is implemented in the frontend of the DefiKicks application. The original code can be found in the DefiKicks Github

const vote = async (proposal: any, voteString: string) => {
  /// ... other code

  const votePhase = 5 * 60;

  const message = {
    proposalId: proposal.id,
    proposalName: proposal.name,
    proposalDescription: proposal.description,
    proposalAdapter: proposal.ipfsHash,
    vote: voteString,
  };

  const messageString = JSON.stringify(message);

  const signature = await library?.getSigner().signMessage(messageString);

  // Encrypt the message using Drand time-lock encryption
  const cyphertext = await timelockEncryption(messageString, 30);

  const vote = {
    proposalId: proposal.id,
    cyphertext,
    account,
    signature,
  };

  newVotes.push(vote);

  console.log("Your vote", JSON.stringify(vote, null, 2));

  await storeVotes(proposal.id, newVotes);
  await notifyNewVote();
};

2. On-chain: Resolution request

Once the off-chain voting phase has ended anyone can request it's resolution by calling the requestVoteResolution function in the Governor smart contract. This function in turn will trigger a Lilypad job that will fetches all the votes and resolve the voting process.

The Lilypad job is defined as docker image that runs a javascript script that fetches all the votes and resolves the voting process. We will go into the details of the script in a later section.

The Lilypad job spec is dinamically built to send the proposal ID as environment variable together with a node url used to fetch the vote power of each voter. The node url is also used to fetch the proposal details from the chain.

See the original smart contract for more details.

    function requestVoteResolution(bytes32 proposalId) public payable virtual {
        require(
            state(proposalId) == ProposalState.ResolutionToRequest,
            "Governor: vote not in ResolutionToRequest state"
        );
        uint256 lilypadFee = bridge.getLilypadFee();
        require(msg.value >= lilypadFee, "Governor: insufficient fee");

        string memory spec = getSpecForProposalId(proposalId);

        uint256 id = bridge.runLilypadJob{value: lilypadFee}(
            address(this),
            spec,
            uint8(LilypadResultType.StdOut)
        );
        require(id > 0, "job didn't return a value");
        proposals[proposalId].bridgeId = id;
        jobIdToProposal[id] = proposalId;
        emit VoteResolutionRequested(proposalId, id);
    }

3.1 Off-chain: Lilypad job execution with Bacalhau

The script run in Bacalhau perform the following steps:

1. Retrieve all the off-chain votes from a decentralised storage (Ceramic stream or IPFS)

2. Decrypt the votes using Drand timelock decryption function

3. Verify the signature of the decrypted vote

4. Compute the vote power of each voter

5. Compute the vote result using the vote power of each voter

6. Compute the inflationary rewards for each voter that voted for the winning option

7. Put the reward information in a Merkle tree and compute the Merkle root together with the proof for each voter

8. Encode all the information in a hex string and console.log it as this is the stdout that will be sent back to the Governor smart contract by the Bacalhau operators that run the job.

Here is the vote decryption and verification code, for the full script see the original script

// other code
const votes = await getVotes(proposal.id);

const decryptedVotes = [];

for (const vote of votes) {
  let message;
  try {
    message = await timelockDecryption(vote.cyphertext);
  } catch (e) {}
  decryptedVotes.push({
    ...vote,
    message,
  });
}

const signedVotes = [];

for (const vote of decryptedVotes) {
  const recoveredAddress = ethers.utils.verifyMessage(
    vote.message,
    vote.signature
  );
  if (recoveredAddress === vote.account) {
    signedVotes.push(vote);
  }
}
// other code

const calldata = ethers.utils.defaultAbiCoder.encode(
  [
    {
      type: "tuple",
      components: [
        { name: "forVotes", type: "uint256" },
        { name: "againstVotes", type: "uint256" },
        { name: "abstainVotes", type: "uint256" },
        { name: "voteMerkleRoot", type: "bytes32" },
        { name: "data", type: "string" },
      ],
    },
  ],
  [
    {
      forVotes,
      againstVotes,
      abstainVotes,
      voteMerkleRoot: tree.root,
      data,
    },
  ]
);

console.log("calldata", calldata);

3.2 On-chain: Lilypad job resolution

Once the job is picked up by the Bacalhau operators and executed, the resolutions are brought on-chain and the Governor smart contract receives a lilypadFulfilled callback from Lilypad.

The _result string contains all the vote resolution information encoded in a hex string. The string is decoded and the resolution information is stored in the smart contract.

For more information on how the result is decoded see the original smart contract

    function lilypadFulfilled(
        address _from,
        uint _jobId,
        LilypadResultType _resultType,
        string calldata _result
    ) external override {
        require(_resultType == LilypadResultType.StdOut);
        require(msg.sender == address(bridge));

        ResolutionResponse memory resolutionResponse = abi.decode(
            hexStrToBytes(_result),
            (ResolutionResponse)
        );
        ProposalCore storage proposal = proposals[jobIdToProposal[_jobId]];
        proposal.voteMerkleRoot = resolutionResponse.voteMerkleRoot;
        proposal.forVotes = resolutionResponse.forVotes;
        proposal.againstVotes = resolutionResponse.againstVotes;
        proposal.abstainVotes = resolutionResponse.abstainVotes;

        emit ProposalUpdated(
            jobIdToProposal[_jobId],
            resolutionResponse.voteMerkleRoot,
            resolutionResponse.forVotes,
            resolutionResponse.againstVotes,
            resolutionResponse.abstainVotes,
            resolutionResponse.data
        );
    }

Using the Lilypad Contracts

1. Create a contract that implements LilypadCallerInterface. As part of this interface you need to implement 2 functions:

   • lilypadFulfilled - a callback function that will be called when the job completes successfully

   • lilypadCancelled - a callback function that will be called when the job fails

2. To trigger a job from your contract, you need to call the LilypadEvents contract which the bridge is listening to. You will connect to Bacalhau network via this bridge. Create an instance of LilypadEvents in your own contract by passing the public contract address above to the LilypadEvents constructor. See Deployed Network Detailsfor address details

3. To make a call to Bacalhau, call runLilypadJob from your function. You need to pass the following parameters:

LilypadEvents Contract

This contract is the bridge between the Bacalhau network and smart contracts & does all the heavy lifting.

To use Lilypad, you only need to take note of one function in this events contract - the runLilypadJob(address _from, string memory _spec, uint8 _resultType) function which takes the following parameters.\

// SPDX-License-Identifier: MIT
pragma solidity >=0.8.4;
import "hardhat/console.sol";
import "@openzeppelin/contracts/utils/Counters.sol";
import "@openzeppelin/contracts-upgradeable/proxy/utils/Initializable.sol";
import "@openzeppelin/contracts-upgradeable/access/AccessControlUpgradeable.sol";
import "@openzeppelin/contracts-upgradeable/proxy/utils/UUPSUpgradeable.sol";
import "./LilypadCallerInterface.sol";

error LilypadEventsUpgradeableError();

/**
    @notice An experimental contract for POC work to call Bacalhau jobs from FVM smart contracts
*/
contract LilypadEventsUpgradeable is Initializable, AccessControlUpgradeable, UUPSUpgradeable {
    bool private initialized;
    bytes32 public constant UPGRADER_ROLE = keccak256("UPGRADER_ROLE");
    
    uint256 public LILYPAD_FEE = 0.03 * 10**18;
    uint256 private escrowAmount = 0;
    uint256 private escrowMinAutoPay  = 5 * 10**18;
    address private escrowAddress = 0x5617493b265E9d3CC65CE55eAB7798796D9108E4; 
    uint256[50] private __gap; //for extra memory slots that may be needed in future upgrades.

    using Counters for Counters.Counter;
    Counters.Counter private _jobIds;

    /// @custom:oz-upgrades-unsafe-allow constructor
    constructor() {
        _disableInitializers();
    }

    function initialize() public initializer {
        require(!initialized, "Contract Instance has already been initialized");
        console.log("Deploying LilypadEvents contract");
        initialized = true;
        __AccessControl_init();
        __UUPSUpgradeable_init();
        _setupRole(DEFAULT_ADMIN_ROLE, msg.sender);
        _grantRole(UPGRADER_ROLE, msg.sender);
    }

    function _authorizeUpgrade(address newImplementation)
        internal
        onlyRole(UPGRADER_ROLE)
        override
    {}

    struct LilypadJob {
        address requestor;
        uint id; //jobID
        string spec; 
        LilypadResultType resultType;
    }
    LilypadJob[] public lilypadJobHistory;

    struct LilypadJobResult {
        address requestor;
        uint id;
        bool success;
        string result;
        LilypadResultType resultType;
    }
    LilypadJobResult[] public lilypadJobResultHistory;
    mapping(address => LilypadJobResult[]) lilypadJobResultByAddress; // jobs by requestor

    /** Events **/
    event NewLilypadJobSubmitted(LilypadJob job);
    event LilypadJobResultsReturned(LilypadJobResult result);
    event LilypadEscrowPaid(address, uint256);

    /** Escrow/ Balance functions **/
    function getEscrowAddress()public view onlyRole(UPGRADER_ROLE) returns(address) {
        return escrowAddress;
    }

    function getEsrowMinAutoPay()public view onlyRole(UPGRADER_ROLE) returns(uint256) {
        return escrowMinAutoPay;
    }

    function setEscrowMinAutoPay(uint256 _minAmount) public onlyRole(UPGRADER_ROLE){
      escrowMinAutoPay = _minAmount;
    }

    function setEscrowWalletAddress(address _escrowAddress) public onlyRole(UPGRADER_ROLE) {
      escrowAddress = _escrowAddress;
    }

    function withdrawBalanceToEscrowAddress() public onlyRole(UPGRADER_ROLE) {
        require(address(this).balance > 0, "No money in contract able to be withdrawn");
        address payable recipient = payable(escrowAddress);
        //ecrowAmount and contract amount Should be the same (if not should be zero'd anyway);
        escrowAmount = 0;
        uint256 amount = address(this).balance;
        recipient.transfer(amount);
        emit LilypadEscrowPaid(recipient, amount);
    }

    function getLilypadFee() public view returns (uint256) {
        return LILYPAD_FEE;
    }

    function getContractBalance() public view onlyRole(UPGRADER_ROLE) returns (uint256) {
      return address(this).balance;
    }

    function setLilypadFee(uint256 _newFee) public onlyRole(UPGRADER_ROLE) {
        LILYPAD_FEE=_newFee;
    }

    /** Lilypad Job via Bacalhau network functions **/
    /** Run your own docker image spec on the network with the _specName = "CustomSpec", You need to pass in the Bacalhau specification for this **/
    function runLilypadJob(address _from, string memory _spec, uint8 _resultType) public payable returns (uint) {
        require(msg.value >= LILYPAD_FEE, "Not enough payment sent to cover job fee");

        uint thisJobId = _jobIds.current();
        LilypadJob memory jobCalled = LilypadJob({
            requestor: _from,
            id: thisJobId,
            spec: _spec,
            resultType: LilypadResultType(_resultType)
        });

        lilypadJobHistory.push(jobCalled);
        emit NewLilypadJobSubmitted(jobCalled);
        _jobIds.increment();

        escrowAmount += msg.value; 
        if(escrowAmount > escrowMinAutoPay){
            uint256 escrowToSend = escrowAmount;
            address payable recipient = payable(escrowAddress);
            //should check contract balance before proceeding
            if(address(this).balance >= escrowToSend) {
              escrowAmount = 0;
              recipient.transfer(escrowToSend);
              emit LilypadEscrowPaid(recipient, escrowToSend);
            }
        }

        return thisJobId;
    }

    // this should really be owner only - our admin contract should be the only one able to call it
    function returnLilypadResults(address _to, uint _jobId, LilypadResultType _resultType, string memory _result) public {
        LilypadJobResult memory jobResult = LilypadJobResult({
            requestor: _to,
            id: _jobId,
            result: _result,
            success: true,
            resultType: _resultType
        });
        lilypadJobResultHistory.push(jobResult);
        lilypadJobResultByAddress[_to].push(jobResult);

        emit LilypadJobResultsReturned(jobResult);
        LilypadCallerInterface(_to).lilypadFulfilled(address(this), _jobId, _resultType, _result);
    }

    function returnLilypadError(address _to, uint _jobId, string memory _errorMsg) public onlyRole(UPGRADER_ROLE) {
        LilypadJobResult memory jobResult = LilypadJobResult({
            requestor: _to,
            id: _jobId,
            success: false,
            result: _errorMsg,
            resultType: LilypadResultType.StdErr
        });
        lilypadJobResultHistory.push(jobResult);
        lilypadJobResultByAddress[_to].push(jobResult);

        emit LilypadJobResultsReturned(jobResult);
        LilypadCallerInterface(_to).lilypadCancelled(address(this), _jobId, _errorMsg);
    }

    function fetchAllJobs() public view returns (LilypadJob[] memory) {
        return lilypadJobHistory;
    }

    function fetchAllResults() public view returns (LilypadJobResult[] memory) {
        return lilypadJobResultHistory;
    }

    function fetchJobsByAddress(address _requestor) public view returns (LilypadJobResult[] memory) {
        return lilypadJobResultByAddress[_requestor];
    }
}

LilypadCaller Interface

This interface ensures that the results of a job run on the Bacalhau network via Lilypad can be returned to the originating contract. This interface needs to be implemented by a smart contract for Lilypad to run.

// SPDX-License-Identifier: MIT
pragma solidity >=0.8.4;

enum LilypadResultType {
  CID,
  StdOut,
  StdErr,
  ExitCode
}

interface LilypadCallerInterface {
  function lilypadFulfilled(address _from, uint _jobId, LilypadResultType _resultType, string calldata _result) external;
  function lilypadCancelled(address _from, uint _jobId, string calldata _errorMsg) external;
}

The cloud is just somebody else's computer...

Overview

This guide will take you through

• Setting up Metamask Wallet for the Lilypad Lalechuza (eth) Testnet

• Funding your wallet with Lilypad Testnet tokens from the faucet.

Install Requirements

Install Docker

Install Lilypad CLI in your terminal / command line

curl -sSL -O https://raw.githubusercontent.com/bacalhau-project/lilypad-modicum/main/lilypad && sudo install lilypad /usr/local/bin/lilypad

Add your private key to your terminal / command line instance:

Get your private key from MetaMask: Accounts -> Account Details -> Show private key

Set your private key in terminal

export PRIVATE_KEY=<the private key you copied>

You can verify you have set it with:

echo $PRIVATE_KEY

Remix

Open the Contract

Alternatively, open the Remix IDE in your browser and copy in the below ExampleClient.sol contract:

Connect to the Testnet

1. Connect MetaMask to the Lalechuza testnet & ensure you have testnet lilETH funds.

2. In the deploy tab in remix [fourth tab in the side bar], ensure you set the environment to "Injected Provider - MetaMask"

Deploy the Contact In remix, compile the ExampleClient.sol contract [third tab in the side bar]

Deploy a new contract by pasting in the Modicum contract address (found here) to the constructor

Deploy an existing ExampleClient.sol contract by using the "At Address" with the following pre-deployed contract address: 0x035C7593D3355b9bE0459dF2296053f887d051f1

Call the runCowsay function

The moment of truth! Let's run the cowsay example!!

Then add the string parameter for what you want the cow to say and click the transact button! Your MetaMask wallet should pop up asking you to confirm the transaction.

Wait a couple of minutes for the job to complete on the compute network. Then you will be able to click the fetchAllResults button to get your IPFS CID result.

Open the string starting with "https://ipfs.io/" in your browser which contains the output of the compute job:

The cowsay result is found under "stdout"

Congrats!

Prefer Video?

Quick Start Guide

Using Lilypad in your own solidity smart contract requires the following steps

1. Create a contract that implements the LilypadCaller interface.

   As part of this interface you need to implement 2 functions:

   • lilypadFulfilled - a callback function that will be called when the job completes successfully

   • lilypadCancelled - a callback function that will be called when the job fails

2. Provide a public Docker Spec compatible for use on Bacalhau in JSON format to the contract.

3. To trigger a job from your contract, you need to call the LilypadEvents contract which the Lilypad bridge is listening to and which connects to the Bacalhau public network. Create an instance of LilypadEvents by passing the public contract address on the network you are using (see Deployed Network Details) to the LilypadEvents constructor.

4. Call the LilypadEvents contract function runLilypadJob() passing in the following parameters.

Implement the LilypadCaller Interface in your contract

Create a contract that implements LilypadCallerInterface. As part of this interface you need to implement 2 functions:

• lilypadFulfilled - a callback function that will be called when the job completes successfully

• lilypadCancelled - a callback function that will be called when the job fails

  /** === LilypadCaller Interface === **/
  pragma solidity >=0.8.4;
  import 'https://github.com/bacalhau-project/lilypad/blob/main/hardhat/contracts/LilypadCallerInterface.sol' //Location of file link

  /** === User Contract Example === **/
  contract MyContract is LilypadCallerInterface {

      function lilypadFulfilled(address _from, uint _jobId,   
          LilypadResultType _resultType, string calldata _result)        
          external override {
          // Do something when the LilypadEvents contract returns    
          // results successfully
      }

      function lilypadCancelled(address _from, uint _jobId, string 
          calldata _errorMsg) external override {
          // Do something if there's an error returned by the
          // LilypadEvents contract
      }
  }

Add a Spec compatible with Bacalhau

Bacalhau operates by executing jobs within containers. This means it is able to run any arbitrary Docker jobs or WASM images

We'll use the public Stable Diffusion Docker Container located here for this example.

Here's an example JSON job specification for the Stable Diffusion job:

{
    "Engine": "docker", 
    "Verifier": "noop",
    "PublisherSpec": {"Type": "estuary"},
    "Docker": {
    "Image": "ghcr.io/bacalhau-project/examples/stable-diffusion-gpu:0.0.1",
    "Entrypoint": ["python", "main.py", "--o", "./outputs", "--p", "A User Prompt Goes here"]},
    "Resources": {"GPU": "1"},
    "Outputs": [{"Name": "outputs", "Path": "/outputs"}],
    "Deal": {"Concurrency": 1}
}

Here's an example of using this JSON specification in solidity:

Note that since we need to be able to add the user prompt input to the spec, it's been split into 2 parts.

string constant specStart = '{'
    '"Engine": "docker",'
    '"Verifier": "noop",'
    '"PublisherSpec": {"Type": "estuary"},'
    '"Docker": {'
    '"Image": "ghcr.io/bacalhau-project/examples/stable-diffusion-gpu:0.0.1",'
    '"Entrypoint": ["python", "main.py", "--o", "./outputs", "--p", "';

string constant specEnd =
    '"]},'
    '"Resources": {"GPU": "1"},'
    '"Outputs": [{"Name": "outputs", "Path": "/outputs"}],'
    '"Deal": {"Concurrency": 1}'
    '}';


//Example of use:
string memory spec = string.concat(specStart, _prompt, specEnd);

Add the Lilypad Events Address & Network Fee

You can do this by either passing it into your constructor or setting it as a variable

// SPDX-License-Identifier: MIT
pragma solidity >=0.8.4;
import "https://github.com/bacalhau-project/lilypad/blob/main/hardhat/contracts/LilypadEventsUpgradeable.sol";
import "https://github.com/bacalhau-project/lilypad/blob/main/hardhat/contracts/LilypadCallerInterface.sol";

/** === User Contract Example === **/
contract MyContract is LilypadCallerInterface {
  address public bridgeAddress; // LilypadEvents contract address for interacting with the deployed LilypadEvents contract
  LilypadEventsUpgradeable bridge; // Instance of the LilypadEvents Contract to interact with
  uint256 public lilypadFee; //=30000000000000000 on FVM;
  
  constructor(address _bridgeContractAddress) {
    bridgeAddress = _bridgeContractAddress; //the LilypadEvents contract address for your network
    bridge = LilypadEventsUpgradeable(_bridgeContractAddress); //create an instance of the Events Contract to interact with
    uint fee = bridge.getLilypadFee(); // you can fetch the fee amount required for the contract to run also
    lilypadFee = fee;
  }

  function lilypadFulfilled(address _from, uint _jobId,   
    LilypadResultType _resultType, string calldata _result)        
    external override {
    // Do something when the LilypadEvents contract returns    
    // results successfully
  }
  
  function lilypadCancelled(address _from, uint _jobId, string 
    calldata _errorMsg) external override {
    // Do something if there's an error returned by the Lilypad Job
  }
}

Call the LilypadEvents runLilypadJob function

Using the LilypadEvents Instance, we can now send jobs to the Bacalhau Network via our contract using the runLilypadJob() function.

In this example we'll use the Stable Diffusion Spec shown above in #add-a-spec-compatible-with-bacalhau

// SPDX-License-Identifier: MIT
pragma solidity >=0.8.4;
import "https://github.com/bacalhau-project/lilypad/blob/main/hardhat/contracts/LilypadEventsUpgradeable.sol";
import "https://github.com/bacalhau-project/lilypad/blob/main/hardhat/contracts/LilypadCallerInterface.sol";

/** === User Contract Example === **/
contract MyContract is LilypadCallerInterface {
  address public bridgeAddress; // Variable for interacting with the deployed LilypadEvents contract
  LilypadEventsUpgradeable bridge;
  uint256 public lilypadFee; //=30000000000000000;
  
  constructor(address _bridgeContractAddress) {
    bridgeAddress = _bridgeContractAddress;
    bridge = LilypadEventsUpgradeable(_bridgeContractAddress);
    uint fee = bridge.getLilypadFee(); // you can fetch the fee amount required for the contract to run also
    lilypadFee = fee;
  }
  
  //** Define the Bacalhau Specification */
  string constant specStart = '{'
      '"Engine": "docker",'
      '"Verifier": "noop",'
      '"PublisherSpec": {"Type": "estuary"},'
      '"Docker": {'
      '"Image": "ghcr.io/bacalhau-project/examples/stable-diffusion-gpu:0.0.1",'
      '"Entrypoint": ["python", "main.py", "--o", "./outputs", "--p", "';

  string constant specEnd =
      '"]},'
      '"Resources": {"GPU": "1"},'
      '"Outputs": [{"Name": "outputs", "Path": "/outputs"}],'
      '"Deal": {"Concurrency": 1}'
      '}';
  

  /** Call the runLilypadJob() to generate a stable diffusion image from a text prompt*/
  function StableDiffusion(string calldata _prompt) external payable {
      require(msg.value >= lilypadFee, "Not enough to run Lilypad job");
      // TODO: spec -> do proper json encoding, look out for quotes in _prompt
      string memory spec = string.concat(specStart, _prompt, specEnd);
      uint id = bridge.runLilypadJob{value: lilypadFee}(address(this), spec, uint8(LilypadResultType.CID));
      require(id > 0, "job didn't return a value");
      prompts[id] = _prompt;
  }

  /** LilypadCaller Interface Implementation */
  function lilypadFulfilled(address _from, uint _jobId,   
    LilypadResultType _resultType, string calldata _result)        
    external override {
    // Do something when the LilypadEvents contract returns    
    // results successfully
  }
  
  function lilypadCancelled(address _from, uint _jobId, string 
    calldata _errorMsg) external override {
    // Do something if there's an error returned by the
    // LilypadEvents contract
  }
}

The following is how to run the "Hello, World!" Job from the Lilypad CLI.

Run Hello, World! Job

• Start Docker

• Open a Terminal window and run the following command:

lilypad run cowsay:v0.0.1 "hello lilypad"

Results:

See the Results

Navigate to the IPFS CID result output in the Results -> https://ipfs.io/ipfs/QmNjJUyFZpSg7HC9akujZ6KHWvJbCEytre3NRSMHzCA6NR

Then click on the stdout folder and you should see the job result!

Setting up Metamask

• Install MetaMask Extension here

• Next, add the Lalechuza testnet chain to MetaMask.

To do this, open MetaMask then click on the network button at the top left of the popup (in the menu bar):

``

Then click the "Add Network" Button.

Next, click on "Add a network manually" at the bottom of the page and enter the Lilypad Testnet details:

Lilypad is a powerful trustless distributed compute platform which leverages key features of blockchain to enable developers to call arbitrary verifiable compute jobs directly from their smart contracts! Lilypad's initial focus is on bringing together the current demand for GPUs (from AI & ML) with decentralised physical infrastructure networks like Filecoin which can supply this demand.

Applications

The ability to perform off-chain decentralised compute over data from smart contracts opens the door to a multitude of possible applications including:

• Inference AI jobs

• ML training jobs

• Invoking & supporting generic ZK computations

• Cross-chain interoperability complement to bridge protocols

• Utilising inbuilt storage on IPFS

• Federated Learning consensus (with Bacalhau insulated jobs)

• IOT & Sensor Data integrations

• Providing a platform for Digital twins

• Supply chain tracking & analysis

• ETL & data preparation jobs

Marketplace

Lilypad 'Layers'

How we see Lilypad network operating on a high level

Testnet Implementations

Lilypad v1 is currently in early testnet phase and is a custom implementation of the ideas & code contained in the paper: "Mechanisms for Outsourcing Computation via a Decentralized Market", which proposed a mediator approach to resolving consensus of deterministic jobs, and also offers insights into running non-deterministic jobs.

Currently, two testnets are functional, both of which allow arbitrary untrusted nodes to join, but use a set of mutually trusted mediators to check jobs using verification by replication (see MODICUM paper for details).

1. Lalechuza - a testnet built on geth

2. Larana - a testnet built on Filecoin IPC (an advanced scaling solution for blockchain that implements a subnet pattern)

This page is a dynamic work in progress! We're working on some better diagrams!

Lilypad Architecture

[upcoming v] Lilypad Layers

[upcoming v] Lilypad Job Flow

[upcoming v] Lilypad Smart Contracts

Lilypad Job Flow

Hackathon & Project Ideas for Lilypad v1

Here's some ideas for what you could build with Lilypad v1!

Module Contributions!

Contribute a Module to Lilypad!

Also...

We're sorry about these docs... we're actively working on it!

More Inspiration?

• Create a gasless transaction pipeline with Bacalhau

• Build new features for Waterlily.ai

Funding your wallet

To obtain funds, connect to the Lilypad Lalechuza network on your wallet and head to the faucet at

Copy your MetaMask wallet address into the bar and click request.

Run Hello, 🐮 World! Job [CLI]

Start Docker

Open a Terminal window and run the following command

Output:

See the Results

Then click on the stdout folder and you should see the job result!

Run Hello, World! Node

To contribute your resources to the network and get paid:

That's it! This will run a Lilypad docker node on your local machine which can accept and run jobs.

Resource Provider Flow

As a resource provider:

• I register my resource offer and send my deposit against that single offer

• The offer is matched with a job and I run the job

• if I ran the job properly - I get my deposit back AND the cost of the job

• if I was caught cheating - I will lose my deposit

This page also comes with a warning:

Overview

Verification on Lilypad v1

There is currently no verification on smart contract-called Lilypad jobs

Verification DOES exist on the CLI version, however.

Mediation on Lilypad v1

How can nodes be trusted to do the compute job?

A resource provider node (compute node), is currently required to put down a deposit of funds before any compute jobs arrive.

On a network of compute nodes, there is also mediator nodes, whose job is to check that nodes in the network are correctly running the submitted jobs (eg. by running the job itself to check for correctness)

Job Pricing

While not every job will take the same amount of time or compute resources to run, currently the network prices all jobs at the same amount. This will change in future versions to more accurately calculate and charge for the compute resource requirements of a job as well as compensating the compute nodes appropriately.

What is Lilypad?

Lilypad serves as a verifiable, trustless, and decentralized computational network engineered to facilitate mainstream adoption of web3 applications. By extending unrestricted, global access to computational power, Lilypad strategically collaborates with decentralized infrastructure networks, such as Filecoin, to formulate a transparent, efficient, and accessible computational ecosystem. While Lilypad does not specifically resolve issues related to the accessibility of AI models, it significantly alleviates challenges associated with procuring high-performance AI hardware. In this context, Lilypad provides decentralized AI computational services. The network recently unveiled its second version, dubbed Lilypad V2 (Aurora), and is actively laying groundwork for multi-chain integration and the deployment of an incentivized test net.

Objective and problem statement

Lilypad aims to mitigate the challenges predominantly associated with the accessibility of high-performance computational hardware. At present, numerous barriers impede developers and organizations from seamlessly integrating projects that require high-performance computing, such as AI technologies, into their applications. Unlike conventional centralized systems, where access to powerful compute hardware is restricted and costly, Lilypad endeavors to democratize this access. Through its verifiable, trustless, and decentralized computational network, Lilypad extends unrestricted, global access to computational power. By leveraging decentralized infrastructure networks such as Filecoin, Lilypad is strategically positioned to enhance the efficiency, transparency, and accessibility of high-performance computing hardware.

Applications

Perform off-chain decentralised compute over data, with on-chain guarantees, and to call this functionality directly from a smart contract, CLI and an easy to use abstraction layer, opens the door to a multitude of possible applications including:

• Inference AI jobs

• ML training jobs

• Invoking & supporting generic ZK computations

• Cross-chain interoperability complement to bridge protocols

• Utilising inbuilt storage on IPFS

• Federated Learning consensus (with Bacalhau insulated jobs)

• IOT & Sensor Data integrations

• Providing a platform for Digital twins

• Supply chain tracking & analysis

• ETL & data preparation jobs

Key features

Some of the key features of Lilypad include:

1. Verifiable Trustless Decentralized Compute Network: Lilypad is a decentralized compute network that aims to provide global, permissionless access to compute power. It leverages decentralized physical infrastructure networks like Filecoin to ensure trustlessness and verifiability.

2. Mainstream Web3 Application Support: Lilypad is designed to enable mainstream web3 applications to use its compute network. It aims to make decentralized AI more accessible, efficient, and transparent for developers and users.

3. Open Compute Network: Lilypad creates an open compute network that allows users to access and run AI models and jobs. It separates module creators from users and curates a set of deterministic modules for users to run, ensuring determinism in verification systems.

4. Multichain Support: Lilypad plans to go multichain, which means it will support multiple blockchain networks. This will increase the scalability and interoperability of the network, allowing users to choose the blockchain that best suits their needs.

5. Incentivized Test Net: Lilypad has plans to launch an incentivized test net, which will provide users with incentives to participate in testing and improving the network. This will help identify and address any issues or challenges before the mainnet launch.

6. Decentralization of Mediators: The team also aims to decentralize the mediators in the network. This means that the decision-making process and governance of the network will be distributed among multiple participants, ensuring a more decentralized and resilient system.

Presentation

In the video above, Ally Haire, Co-Founder of Lilypad, will provide an introduction to the project and its goals.

Overview

The Filecoin Data Prep Module is designed to chunk data into CAR files from an S3 bucket - hence the name, since it prepares the data to be uploaded to Filecoin. The repo for this module can be found here.

[CLI] Usage

Run the module

lilypad run filecoin_data_prep:v0.0.1 '{"ipfs_cid": "CID"}'

Execution:

Results:

Full Module

Overview

This LLM Inference Module is a community-contributed module developed at AugmentHack.xyz The repo for this module can be found here.

See the original AugmentHack entry below:

[CLI] Usage

Usage:

lilypad run fastchat:v0.0.1 "paramsStr"

Inputs:

Where "paramsStr" is a question in CID form for the LLM. For example https://ipfs.io/ipfs/QmcPjQwVcJiFge3yNjVL2NoZsTQ3GBpXAZe21S2Ncg16Gt is a bare file CID which contains

{
    "template": "You are a friendly chatbot assistant that responds conversationally to users' questions. \n Keep the answers short, unless specifically asked by the user to elaborate on something. \n \n Question: {question} \n \n Answer:",
    "parameters": {"question": "What is a chatbot?"}
}

To use it you would run:

lilypad run fastchat:v0.0.1 QmcPjQwVcJiFge3yNjVL2NoZsTQ3GBpXAZe21S2Ncg16Gt

Outputs:

The output will be an IPFS CID, for example running the above input would result in the following link:

https://ipfs.io/ipfs/QmVNXCAfJgER6U7Z5XT8QaAVFPdwmtSFE6c9sUaAx7ttZs

Under link/output/result.json you will see

{"question": "What is a chatbot?", "text": "<pad> A  chatbot  is  a  computer  program  that  can  interact  with  users  in  a  conversational  manner.  It  is  designed  to  answer  questions  and  provide  information  in  a  way  that  is  natural  and  conversational.\n"}

LLM Module Code

Overview

Generically, stable diffusion is what happens when you put a couple of drops of dye into a bucket of water. Given time, the dye randomly disperses and eventually settles into a uniform distribution which colours all the water evenly

In computer science, you define rules for your (dye) particles to follow and the medium this takes place in.

Stable Diffusion is a machine learning model used for text-to-image processing (like Dall-E) and based on a diffusion probabilistic model that uses a transformer to generate images from text. There are several open-source stable diffusion models out there (made famous by Stability.ai) and they continue to improve and become even more fully featured - SDXL0.9 is one of the more recently open-sourced models.

[CLI] Running Stable Diffusion SDXL 0.9

To run stable diffusion use the SDXL module like so:

lilypad run sdxl:v0.9-lilypad1 "an astronaut riding on a unicorn"

The output will look like this:

Take the ipfs link given in the results and paste it into your browser:

In the /outputs folder, you'll find the image:

Since modules are deterministic, running this command with the same text prompt will produce the same image, since the same seed is also used (the default seed is 0).

To change the image, you can pass in a different seed number:

lilypad run sdxl:v0.9-lilypad1 '{"prompt": "an astronaut riding on a unicorn", "seed": 9}'

CLI Video

[Smart Contract] Running Stable Diffusion SDXL 0.9

To trigger the SDXL0.9 module from a smart contract, firstly you need to create your own client contract to call the module from. In order to receive results back from the Lilypad network, you will also need to 1. Connect to the Lilypad Modicum Contract (and create an instance of it in your own contract using the current address found here) 2. Implement the Modicum Contract receiveJobResults() interface.

// SPDX-License-Identifier: GPLv3
pragma solidity ^0.8.6;

// give it the ModicumContract interface 
// NB: this will be a separate import in future.
interface ModicumContract {
  function runModuleWithDefaultMediators(string calldata name, string calldata params) external payable returns (uint256);
}

contract SDXLCaller {
  address public contractAddress;
  ModicumContract remoteContractInstance;
  
  // See the latest result.
  uint256 public resultJobId;
  string public resultCID;

  // The Modicum contract address is found here: https://github.com/bacalhau-project/lilypad-modicum/blob/main/latest.txt
  // Current: 0x422F325AA109A3038BDCb7B03Dd0331A4aC2cD1a
  constructor(address _modicumContract) {
    require(_modicumContract != address(0), "Contract cannot be zero address");
    contractAddress = _modicumContract;
    //make a connection instance to the remote contract
    remoteContractInstance = ModicumContract(_modicumContract);
  } 

  /*
  * @notice Run the SDXL Module
  * @param prompt The input text prompt to generate the stable diffusion image from
  */
  function runSDXL(string memory prompt) public payable returns (uint256) {
    require(msg.value == 2 ether, "Payment of 2 Ether is required"); //all jobs are currently 2 lilETH
    return remoteContractInstance.runModuleWithDefaultMediators{value: msg.value}("sdxl:v0.9-lilypad1", prompt);
  }
  
  // This must be implemented in order to receive the job results back!
  function receiveJobResults(uint256 _jobID, string calldata _cid) public {
    resultJobId =_jobID;
    resultCID = _cid;
  }

}

NB: You could also add the seed as a parameter to run this.

return remoteContractInstance.runModuleWithDefaultMediators{value: msg.value}("sdxl:v0.9-lilypad1",`` params);

Remix

Try it yourself!

1. Ensure your MetaMask wallet is set to the Lalechuza testnet and has lilETH testnet funds from the faucet.

2. Set the remix environment to "Injected Provider - MetaMask" (& ensure MetaMask has the lalechuza chain selected)

3. Then - Deploy a new contract passing in the Modicum Contract address found here OR - Open the contract at this example address: 0x31e7bF121EaB1C0B081347D8889863362e9ad53A

1. Call the runSDXL Module, passing in a prompt and sending 2 lilETH in the value field. Your MetaMask wallet should pop up for you to confirm the payment and transaction.

1. Give it some time and check the resultCID variable. You can then open this result in your browser with https://ipfs.io/ipfs/<resultCID> or ipfs://<resultCID> in IPFS compatible browsers like Brave.

Video

SDXL Module Code

Find the SDXL module code here. There's also a generic Stable Diffusion module here.

Overview

[CLI] Running DuckDB Module

DuckDB Module Code

Overview

Contributing your own module to use on Lilypad is possible and welcome! Essentially modules on Lilypad currently operate like Job Specifications do - take a look at this page on to see more about how this works.

Requirements

1. Modules should be either Docker or WASM Images that align to the Bacalhau job specifications (which then have some added metadata that defines the runtime options of the module).

Module Example

Here is an example of the SDXL module in python:

It's a function that given a "string" will return a docker job spec. That string can be whatever you want, JSON, csv, raw LLM prompt etc. In terms of output directories - you can see the SDXL example linked above names the "/outputs" folder and then will use that path in the command inside the Docker container. Any named folders like this that the Docker image can write files into - will be included as part of the results Lilypad gets back out of the job (and will change the result hash)

Using Bacalhau as a testing ground

It's currently advisable to develop your module with first (because it's FAR easier to get setup as a development environment than Lilypad is currently). If you can get a function like the one shown above that will, given a string, write a bacalhau job spec that you can test with the bacalhau CLI - then you have already done 98% of the Lilypad module

To add this to Lilypad, submit a PR which also includes this file .

Contributing Guidelines

The Future of Community Contributed Modules

From Lilypad incentivised testnet [Q4 2023] onwards (and perhaps even earlier), its probable modules contributed by community members will be eligible for a % fee of jobs that run these modules in order to encourage the growth of the module ecosystem on the Lilypad Network.

This page is a dynamic work in progress! We're working on some better diagrams!

Lilypad Technical Architecture

Based on Academic Research

The architecture of Lilypad is inspired by the research paper titled "Mechanisms for Outsourcing Computation via a Decentralized Market." The paper introduces MODiCuM, a decentralized system that allows for computational outsourcing in an open market. Just like MODiCuM, Lilypad aims to create an open market of computational resources by introducing various decentralized services like solver, resource provider, job creator, mediator, and directory services. MODiCuM's unique approach to deterring misbehavior in a decentralized environment through dedicated mediators and enforceable fines has influenced Lilypad's own design, particularly in the areas of dispute resolution and system integrity.

Abstract of the paper: Mechanisms for Outsourcing Computation via a Decentralized Market As the number of personal computing and IoT devices grows rapidly, so does the amount of computational power that is available at the edge. Since many of these devices are often idle, there is a vast amount of computational power that is currently untapped, and which could be used for outsourcing computation. Existing solutions for harnessing this power, such as volunteer computing (e.g., BOINC), are centralized platforms in which a single organization or company can control participation and pricing. By contrast, an open market of computational resources, where resource owners and resource users trade directly with each other, could lead to greater participation and more competitive pricing. To provide an open market, we introduce MODiCuM, a decentralized system for outsourcing computation. MODiCuM deters participants from misbehaving-which is a key problem in decentralized systems-by resolving disputes via dedicated mediators and by imposing enforceable fines. However, unlike other decentralized outsourcing solutions, MODiCuM minimizes computational overhead since it does not require global trust in mediation results. We provide analytical results proving that MODiCuM can deter misbehavior, and we evaluate the overhead of MODiCuM using experimental results based on an implementation of our platform.

Developer Experience and Determinism: A Talk by Luke Marsden

In this insightful presentation, Luke Marsden, a core member of the Lilypad project, delves into the complex relationship between determinism and developer experience. He highlights the necessity for determinism in verification systems and discusses the challenges of implementing it during runtime. The talk also covers the project's strategy to separate module creators from end-users and to provide a curated list of deterministic modules. Key points in the roadmap like improving user experience, decentralizing mediators, and automating determinism checks are also touched upon. The presentation concludes with a live demo of running AI jobs on Lilypad V2.

The cloud is just somebody else's computer...

Overview

This guide will take you through

Install Requirements

Install CLI

1. With GO toolchain

You may then need to set:

2. Via officially released binaries

You may then need to set:

Verifying if the install is successful. Execute lilypad on your terminal and it should produce the following response.

Note: Make sure to you have Lilypad CLI installed.

Note: Make sure you have SERVICE_SOLVER, SERVICE_MEDIATORS and WEB3_PRIVATE_KEY env variables in your environment

Run Cowsay

1. Run the command below

1. Wait for the compute to take place and for the results to be published

1. Viewing your results

Overview of LoRA Fine Tuning

LoRA stands for Low Rank Adaptation and is a mathematical technique to reduce the number of parameters that are trained in a model - so instead of fine-tuning all the weights that constitute the weight matrix of the pre-trained large language model, two smaller matrices that approximate this larger matrix are fine-tuned.

What this means in practice is that instead of needing to build a custom model off all the original input data material (and needed many many GPUs to do so), LoRA means you can fine-tune an existing model (such as SDXL 0.9 Stable Diffusion) to be biased towards a certain result on just one GPU.

For example, an open source Stable Diffusion model can be fine-tuned to produce images in the style of Claude Monet paintings using LoRA. Fun fact: This is how trains artist models - look how good the results are even without an up to date Stable Diffusion model like SDXL0.9!

[CLI] Running LoRA Fine Tuning [coming soon]

To run a LoRA fine-tuning job, just provide the training data for the job to the command:

NB: the params above should be yaml eg. {seed: 42, 'images_cid': 'Qm...'} where images_cid contains an images.zip with training images in it.

This will output a result model CID, which can then be used to generate new images in this particular style:

[Smart Contract] Running LoRA Fine Tuning [coming soon]

LoRA Module Code

Funding your wallet

To obtain funds, connect to the lilypad v2 Aurora testnet network on your wallet and head to the faucet at http://faucet.lilypad.tech to get ETH and LP

Copy your metamask wallet address into the bar and click request.

Overview

Generically, stable diffusion is what happens when you put a couple of drops of dye into a bucket of water. Given time, the dye randomly disperses and eventually settles into a uniform distribution which colours all the water evenly

In computer science, you define rules for your (dye) particles to follow and the medium this takes place in.

Stable Diffusion is a machine learning model used for text-to-image processing (like Dall-E) and based on a diffusion probabilistic model that uses a transformer to generate images from text. There are several open-source stable diffusion models out there (made famous by Stability.ai) and they continue to improve and become even more fully featured - SDXL0.9 is one of the more recently open-sourced models.

[CLI] Running Stable Diffusion SDXL 0.9

To run stable diffusion use the sdxl module like so:

The output will look like this:

Take the ipfs link given in the results and paste it into your browser:

You could also check the output folder that would have been downloaded at the end of running the job

In the /outputs folder, you'll find the image:

Since modules are deterministic, running this command with the same text prompt will produce the same image, since the same seed is also used (the default seed is 0).

To change the image, you can pass in a different seed number:

CLI Video

In this video, Sam demos the use of SDXL in the Lilypad network.

Looking for Smart Contracts?

Try this to start!

Getting LilyPad Tokens

To acquire LilyPad tokens, follow these steps:

1. Visit the Lilypad Faucet at

2. Enter your wallet address.

3. Receive some LP tokens.

Setting up Environment Variables

Set the environment variables as shown below:

Running cowsay

Note: Ensure that you have Lilypad installed before proceeding.

To execute cowsay, use the following command:

Output:

See the Results

1. by navigating to the local folder

Here, you can view the stdout and stderr as well as the outputs folder for the run