Building an SDK v0.1.24 – Balancer Relayers and Pool Migrations

What Is A Relayer?

A relayer is a contract that is authorized by the protocol and users to make calls to the Vault on behalf of the users. It can use the sender’s ERC20 vault allowance, internal balance and BPTs on their behalf. Multiple actions (such as exit/join pools, swaps, etc) can be chained together which improves the UX.

For security reasons a Relayer has to be authorised by the Balancer DAO before it can be used (see previous votes for V1 and V2) and even after authorisation each user would still be required to opt into the relayer by submitting an approval transaction or signing a message.

How It Works

Contracts

The Balancer Relayers are composed of two contracts, BalancerRelayer, which is the single point of entry via the multicall function and a library contract, such as the V3 VaultActions, which defines the allowed behaviour of the relayer, for example – VaultActions, LidoWrapping, GaugeActions.

Having the multicall single point of entry prevents reentrancy. The library contract cannot be called directly but the multicall can repeatedly delegatecall into the library code to perform a chain of actions.

Some psuedo code demonstrating how an authorisation, exitPool and swap can be chained and called via the multicall function:

const approval = buildApproval(signature); // setRelayerApproval call
const exitPoolCallData = buildExitPool(poolId, bptAmt); // exitPool call
const swapCallData = buildSwap(); // batchSwap call

const tx = await relayer.multicall([approval, exitPoolCallData, swapCallData]);

Approval

A user has to approve each Relayer before they can use it. To check if a Relayer is approved we can use hasApprovedRelayer on the Vault:

const isApprove = await vault.hasApprovedRelayer(userAddress, relayerAddress)

And we can grant (or revoke) approval for a given relayer by using setRelayerApproval:

const approvalTx = await vault.setRelayerApproval(userAddress, relayerAddress, isApprove);

A Relayer can also be approved by using the setRelayerApproval function from the BaseRelayerLibrary contract. Here a signed authorisation message from the user is passed as an input parameter. This allows an approval to be included at the start of a chain of actions so the user only needs to submit a single transaction creating a better UX.

Chained References

Output References allow the Relayer to store output values from once action which can then be read and used in another action. This allows us to chain together actions. For example we could exit a pool, save the exit amounts of each token to a reference and then do a batchSwap using the references as input amounts for each swap:

An OutputReference consists of an index and a key:

struct OutputReference {
  uint256 index;
  uint256 key;
}

Where the key is the slot the value will be stored at. Index indicates which output amount should be stored. For example if exitPool exits to 3 tokens, DAI (index 0), USDC (1), USDT (2), we would want to use index 0 to store DAI, 1 for USDC, etc.

Example Use Case – Pool Migration

Intro

Balancer aims for the best capital efficiency for LPs so it made sense to offer the option to migrate from the old “staBal3” pool consisting of DAI, USDC and USDT to a new “boosted” stable pool which is more capital efficient because it uses yield bearing assets.

To migrate between these pools would take multiple steps:

  1. unstake from staBal3 gauge → staBalBpt
  2. exitPool from staBal, staBalBpt → DAI, USDC, USDT
  3. join the bb-a-usd2 pool by using batchSwaps
    1. DAI → bbausd2Bpt
    2. USDC → bbausd2Bpt
    3. USDT → bbausd2Bpt
  4. stake bbausd2Bpt in gauge

This would be quite an ordeal for a user to do manually but the Relayer can be used to combine all these actions into a single transaction for the user.

Details

As this is a well defined one off action we decided to add this function to the SDK as a “Zap” under a Migrations module. The user can call the staBal3 function to get all the call data required to call the tx:

{ to, data } = migrations.stabal3(
  userAddress,
  staBal3Amount,
  minBbausd2Out,
  isStaked,
  authorisationSignature
);

Behind the scenes all the call data for each step is crafted and the encoded multicall data is returned:

calls = [
        this.buildSetRelayerApproval(authorisation),
        this.buildWithdraw(userAddress, staBal3Amount),
        this.buildExit(relayer, staBal3Amount),
        this.buildSwap(minBbausd2Out, relayer),
        this.buildDeposit(userAddress),
      ];

const callData = balancerRelayerInterface.encodeFunctionData('multicall', [
      calls,
    ]);

buildSetRelayerApproval allows the user to pass the approval signature if this is their first time using the relayer. This allows us to approve and execute the migration all in a single transaction.

buildWithdraw and buildDeposit handle the gauge actions. The initial call is to withdraw from the staBal gauge and the final call deposits the bbausd2 bpt into the new gauge. We withdraw directly to the Relayer address rather than the users. The gauges return the tokens to the caller, so sending them to the user costs more as we need to manually transfer them:

gauge.withdraw(amount);
// Gauge does not support withdrawing BPT to another address atomically.
// If intended recipient is not the relayer then forward the withdrawn BPT on to the recipient.
if (recipient != address(this)) {
    IERC20 bptToken = gauge.lp_token();
    bptToken.transfer(recipient, amount);
}

Skipping this has two benefits. Firstly it saves gas by avoiding an extra transfer. It also avoids approval issues as now the Relayer is just using its own funds. The final deposit uses the userAddress to send the staked tokens from the Relayer back to the user.

buildExit creates the exitPool call:

// Ask to store exit outputs for batchSwap of exit is used as input to swaps
    const outputReferences = [
      { index: assetOrder.indexOf('DAI'), key: EXIT_DAI },
      { index: assetOrder.indexOf('USDC'), key: EXIT_USDC },
      { index: assetOrder.indexOf('USDT'), key: EXIT_USDT },
    ];

    const callData = Relayer.constructExitCall({
      assets,
      minAmountsOut: ['0', '0', '0'],
      userData,
      toInternalBalance: true,
      poolId: this.addresses.staBal3.id,
      poolKind: 0, // This will always be 0 to match supported Relayer types
      sender,
      recipient: this.addresses.relayer,
      outputReferences,
      exitPoolRequest: {} as ExitPoolRequest,
    });

Output references are used to store the final amounts of each stable token received from the pool. We have precomputed the keys by using the Relayer.toChainedReference helper, like:

const EXIT_DAI = Relayer.toChainedReference('21');
const EXIT_USDC = Relayer.toChainedReference('22');
const EXIT_USDT = Relayer.toChainedReference('23');

These will be used later as inputs to the swaps.

Also of interest is the fact we set toInternalBalance to true. The Balancer V2 vault can accrue ERC20 token balances and keep track of them internally in order to allow extremely gas-efficient transfers and swaps. Exiting to internal balances before the swaps allows us to keep gas costs down.

Because we have previously exited into internal balances we also don’t have to worry about the users having previously approved the Relayer for the tokens:

if (fromInternalBalance) {
// We take as many tokens from Internal Balance as possible: any remaining amounts will be transferred.
uint256 deductedBalance = _decreaseInternalBalance(sender, token, amount, true);
// Because deductedBalance will be always the lesser of the current internal balance
// and the amount to decrease, it is safe to perform unchecked arithmetic.
amount -= deductedBalance;
}

if (amount > 0) {
token.safeTransferFrom(sender, address(this), amount);
}

so the amount will be 0 and the safeTransferFrom call will not be executed.

buildSwap – We can join bbausd2 using a swap thanks to the PhantomBpt concept so here we create a batchSwap call that swaps each stable token to the bbausdBpt and we use the output references from the exitPool call as the input amounts to the swap (which is great as we don’t need to precompute these).

const swaps: BatchSwapStep[] = [
    {
      poolId: this.addresses.linearDai2.id,
      assetInIndex: 1,    // DAI
      assetOutIndex: 2,   // bDAI
      amount: EXIT_DAI.toString(),
      userData: '0x',
    },
    {
      poolId: this.addresses.bbausd2.id,
      assetInIndex: 2,  // bDAI
      assetOutIndex: 0,  // bbausd2
      amount: '0',
      userData: '0x',
    }
    ...
    {
      poolId: this.addresses.linearUsdc2.id,
      assetInIndex: 3,  // USDC
      assetOutIndex: 4, // bUSDC
      amount: EXIT_USDC.toString(),
      userData: '0x',
    },
    ...

In the Relayer VaultActions contract we can see how the swap amounts are set to the value stored in the reference:

for (uint256 i = 0; i < swaps.length; ++i) {
	uint256 amount = swaps[i].amount;
  if (_isChainedReference(amount)) {
	  swaps[i].amount = _getChainedReferenceValue(amount); //e.g. EXIT_DAI
  }
}

And finally (😅) we use another output reference to store the total amount out of bbausd2:

const outputReferences = [{ index: 0, key: SWAP_RESULT_BBAUSD }];

This is used as an input to the final gauge deposit to make sure we stake all the BPT that we have received and that should conclude the migration! You can see this in action on a local fork (yay no real funds required!) by running the integration test here.

Conclusion

The Balancer Relayer is probably not that well known so hopefully this has given a good overview of some of its functionality and flexibility. There’s a lot of room for experimentation and improvement of UX for complex operations so its worth investigating!

Photo by Austrian National Library on Unsplash

Building an SDK 0.1.14 – Adding a Contracts module

Intro

The idea of adding this was to make accessing Balancer contracts easier for users. Normally you need to find and import ABIs and deal with deployment addresses, if we want to make it easy we should just remove that complexity.

Also we are trying to make the main SDK functions return the contract name and functions as part of the attributes returned. This means the user could then just call using something like:

const { contractName, functionName, attributes } = transactionAttributes;

sdk.contracts[contractName][functionName](attributes)

Typechain

Typechain is a package that provides TypeScript bindings for Ethereum contracts. This means functions are statically typed and there is also IDE support which makes things safer and easier to develop against.

Balancer has its own @balancer-labs/typechain package that exposes instances of the commononly used contracts. Adding this to the SDK means we can remove the need to import ABI jsons and we can now create instances of contracts by doing:

import {
    Vault__factory
} from '@balancer-labs/typechain';

Vault__factory.connect(
            this.networkConfig.addresses.contracts.vault,
            provider
        );

which will return a typed Vault contract.

Module

  • Uses BALANCER_NETWORK_CONFIG and config.network to find vault/lidoRelayer/multicall addresses.
  • Added contracts getter to SDK module:
constructor(
        public config: BalancerSdkConfig,
        public sor = new Sor(config),
        public subgraph = new Subgraph(config),
        public pools = new Pools(config),
        public balancerContracts = new Contracts(config, sor.provider)
    ) { ... }

get contracts(): ContractInstances {
        return this.balancerContracts.contracts;
    }

This can then be called like:

const vaultContract = balancer.contracts['vault'];

or:

const vaultContract = balancer.contracts.vault

which will provide typed function refs.

Tradeoffs

One interesting discussion is the trade off of using the Contracts module within other modules. As of now only the Swaps and Multicaller modules using contracts. Using the Contracts module means we either have to pass Contracts in constructor, which adds an extra step if someone want to use modules independently:

const contracts = new Contracts(config)
const swaps = new Swaps(config, contracts)

or we instantiate Contracts within the module – which ends up happening twice if we use the high level SDK function as it is also instantiated there. For now we have decided to use the Typechain factories to instantiate the contracts within the module and will revisit in future if needed.

Photo by Pablo Arroyo on Unsplash

Node Performance Measurement

I was working on optimising a javascript maths function and wanted to compare the performance of different versions of the code. Initially I had some difficulty because I was approaching it incorrectly so I wanted to make a note for future reference.

First mistake – only using a small number of runs. I was comparing the two different functions with a very small iteration amount. This was leading to weird results where each iteration would get faster even though it was actually the same code running. I think this was related to some compiler optimisation or something. Anyway, after reading JavaScript Compiler Optimization Techniques, I changed to a very large number of runs. This made the results much more consistent.

Second mistake – using perf_hooks incorrectly. From the same blog I also found out the nice way to use perf_hooks to measure the performance:`

import { performance, PerformanceObserver } from 'perf_hooks';

let iterations = 1_000_000;

performance.mark('start');
while (iterations--) {
    StableMath._calcOutGivenIn(
        poolPairDataBigInt.amp,
        poolPairDataBigInt.balances,
        poolPairDataBigInt.tokenIndexIn,
        poolPairDataBigInt.tokenIndexOut,
        amt,
        poolPairDataBigInt.fee
    );
}
performance.mark('end');

iterations = 1_000_000;

performance.mark('startSecond');
const invariant = StableMath._calculateInvariant(
    poolPairDataBigInt.amp,
    poolPairDataBigInt.balances,
    true
);
while (iterations--) {
    StableMath._calcOutGivenInNoInv(
        poolPairDataBigInt.amp,
        poolPairDataBigInt.balances,
        poolPairDataBigInt.tokenIndexIn,
        poolPairDataBigInt.tokenIndexOut,
        amt,
        poolPairDataBigInt.fee,
        invariant
    );
}
performance.mark('endSecond');

const obs = new PerformanceObserver((list, observer) => {
    console.log(list.getEntries()); // [0]);
    performance.clearMarks();
    observer.disconnect();
});
obs.observe({ entryTypes: ['measure'] });

performance.measure('NoOptimisation', 'start', 'end');
performance.measure('WithOptimisation', 'startSecond', 'endSecond');

Which results in an output like:

[
  PerformanceMeasure {
    name: 'NoOptimisation',
    entryType: 'measure',
    startTime: 2369.287365913391,
    duration: 35891.85489702225,
    detail: null
  },
  PerformanceMeasure {
    name: 'WithOptimisation',
    entryType: 'measure',
    startTime: 38261.19673395157,
    duration: 18529.005373954773,
    detail: null
  }
]

As well as having a nice output it also shows my optimisation worked pretty nicely!

Photo by Saffu on Unsplash

Forking Brilliant

Houston We Have A Problem

I want to check that a transaction will work on mainnet for an account I don’t control. In this case it’s for a large LP in the Balancer staBal3 pool and I want to check they could successfully migrates their staBal3 to the new bb-a-USD using a Relayer multicall with the params created by the SDK.

This definitely isn’t the most elegant way of doing things but it works!

Whale Hunting

The first thing I need to do is to find a large staBal3 LP account and figure out their BPT balance. I can use the Balancer Subgraph to query account pool shares for the staBal3 pool. Query looks like:

query MyQuery {
  poolShares(where: {poolId: "0x06df3b2bbb68adc8b0e302443692037ed9f91b42000000000000000000000063"}, orderBy: balance, orderDirection: desc) {
    id
    balance
  }
}

I then manually worked my way down the list of addresses until I found one that was an EOA: https://etherscan.io/address/0x4086e3e1e99a563989a9390facff553a4f29b6ee and at the time of investiagation this had a BPT balance of: 10205792.037653741889318463 (a cool $10.28mil!).

Exit Stage Right

The SDK helper function (relayer.exitPoolAndBatchSwap) that creates the call data requires an input param of expectedAmountsOut which in this case represents the DAI/USDC/USDT amounts out when exiting the staBal3 pool. Because I don’t have the maths required for this exposed yet a quick way to get this is to see the output amounts using Withdraw in the UI. There’s a very nice tool that allows us to simulate this when we don’t have control of the account of interest: https://www.impersonator.xyz/

Now that I’ve got all the info required I can generate the call data by using the helper function. In this case we get an array of call data which represent an exitPool call on staBal3 followed by a batchSwap that swaps the stables received from the exit to the bb-a-USD BPT.

The Magic

Tenderly has lots of useful features including Transaction Simulations. To begin I tried simulating the multicall call on the Mainnet Relayer but the tx failed highlighting a new issue – the account hasn’t approved the Balancer Relayer. To get around this I can use a Tenderly Fork – “Forks allow you to chain simulation and test out complex scenarios with live on-chain data”. This is cool because I can now fork the chain make an approval on the relayer from the account which then allows me to succesfully simulate the multicall!

Photo by Joel Muniz on Unsplash

Building an SDK 0.1.0 – Improving SOR data sourcing

Intro

A big focus for Balancer Labs this year is to make it really easy to build on top of the protocol. To aid in that we’re putting together the `@balancer-labs/sdk npm package. As the current lead in this project I thought I’d try and document some of the work to help keep track of the changes, thought process and learning along the way. It’ll also be useful as a reminder of what’s going on!

SOR v2

Some background

We already have the Smart Order Router (@balancer-labs/sor), a package that devs can use to source the optimal routing for a swap using Balancer liquidity. It’s used in Balancers front-end and other projects like Copper and is a solver for Gnosis BGP. It’s also used in the Beethoven front-end (a Balancer friendly fork on Fantom, cool project and team and worth checking out).

The SOR is also used and exposed by the SDK. It’s core to making swaps accesible but is also used for joining/exiting Boosted Pools which uses PhantomBpt and swaps (a topic for another time I think!).

SOR Data

The diagram below shows some of the core parts of the SOR v2.

SOR v2

To choose the optimal routes for a swap the SOR needs information about the Balancer pools and the price of assets. And as we can see from the diagram the sourcing of this data is currently very tightly coupled to the SOR. Pools data is retrieved from the Subgraph and updated with on-chain balances using a multicall. And asset pricing is retrieved from CoinGecko.

Recently Beethoven experienced a pretty large growth spurt and found there were some major issues retrieving data from the Subgraph. They also correctly pointed out that CoinGecko doesn’t always have the asset pricing (especially on Fantom) and this information could be available from other sources.

After some discussions with Daniel (a very helpful dev from Beethoven) it was agreed that a good approach would be to refactor the SOR to create composability of data fetching so the user is able to have more control over where data is coming from. With this approach, the SOR doesn’t need to know anything about CoinGecko or the Subgraph and the data could now come from anywhere (database, cache, on chain, etc.), and as long as it implements the interface, the SOR will work properly.

Changes – SOR v3

I came back from Christmas break and Daniel had made all the changes – friendly forks for the win 💪! The interface changes are breaking but the improvements are worth it – SOR 3.0.0.

Config

The goal was to remove all the chain specific config from the SOR and pass it in as a constructor parameter. This helps to avoid non-scalable hard-coded values and encorages a single source of truth. It also gives more flexibility for the variables and makes the code easier to test.

There is now the SorConfig type:

export interface SorConfig {
    chainId: number;
    weth: string;
    vault: string;
    staBal3Pool?: { id: string; address: string };
    wethStaBal3?: { id: string; address: string };
    usdcConnectingPool?: { id: string; usdc: string };
}

Pool Data

The goal here is to allow for flexibility in defining where the pool data is fetched from. We define a generic PoolDataService that has a single function getPools, which serves as a generic interface for fetching pool data. This allows allow for any number of custom services to be used without having to change anything in the SOR or SDK.

export interface PoolDataService {
    getPools(): Promise<SubgraphPoolBase[]>;
}

Approaching it this way means all the Subgraph and on-chain/multicall fetching logic is removed from the SOR. These will be added to the Balancer SDK as stand-alone services. But as a simple example this is a PoolDataService that retrieves data from Subgraph:

export class SubgraphPoolDataService implements PoolDataService {
    constructor(
        private readonly chainId: number,
        private readonly subgraphUrl: string
    ) {}

    public async getPools(): Promise<SubgraphPoolBase[]> {
        const response = await fetch(this.subgraphUrl, {
            method: 'POST',
            headers: {
                Accept: 'application/json',
                'Content-Type': 'application/json',
            },
            body: JSON.stringify({ query: Query[this.chainId] }),
        });

        const { data } = await response.json();

        return data.pools ?? [];
    }
}

Asset Pricing

The goal here is to allow for flexibility in defining where token prices are fetched from. We define a generic TokenPriceService that has a single function getNativeAssetPriceInToken. Similar to the PoolDataService this offers flexibility in the service that can be used, i.e. CoingeckoTokenPriceService or SubgraphTokenPriceService.

export interface TokenPriceService {
    /**
     * This should return the price of the native asset (ETH) in the token defined by tokenAddress.
     * Example: BAL = $20 USD, ETH = $4,000 USD, then 1 ETH = 200 BAL. This function would return 200.
     * @param tokenAddress
     */
    getNativeAssetPriceInToken(tokenAddress: string): Promise<string>;
}

All the CoinGecko code is removed from the SOR (to be added to SDK). An example TokenPriceService using CoinGecko:

export class CoingeckoTokenPriceService implements TokenPriceService {
    constructor(private readonly chainId: number) {}

    public async getNativeAssetPriceInToken(
        tokenAddress: string
    ): Promise<string> {
        const ethPerToken = await this.getTokenPriceInNativeAsset(tokenAddress);

        // We get the price of token in terms of ETH
        // We want the price of 1 ETH in terms of the token base units
        return `${1 / parseFloat(ethPerToken)}`;
    }

    /**
     * @dev Assumes that the native asset has 18 decimals
     * @param tokenAddress - the address of the token contract
     * @returns the price of 1 ETH in terms of the token base units
     */
    async getTokenPriceInNativeAsset(tokenAddress: string): Promise<string> {
        const endpoint = `https://api.coingecko.com/api/v3/simple/token_price/${this.platformId}?contract_addresses=${tokenAddress}&vs_currencies=${this.nativeAssetId}`;

        const response = await fetch(endpoint, {
            headers: {
                Accept: 'application/json',
                'Content-Type': 'application/json',
            },
        });

        const data = await response.json();

        if (
            data[tokenAddress.toLowerCase()][this.nativeAssetId] === undefined
        ) {
            throw Error('No price returned from Coingecko');
        }

        return data[tokenAddress.toLowerCase()][this.nativeAssetId];
    }

    private get platformId(): string {
        switch (this.chainId) {
            case 1:
                return 'ethereum';
            case 42:
                return 'ethereum';
            case 137:
                return 'polygon-pos';
            case 42161:
                return 'arbitrum-one';
        }

        return '2';
    }

    private get nativeAssetId(): string {
        switch (this.chainId) {
            case 1:
                return 'eth';
            case 42:
                return 'eth';
            case 137:
                return '';
            case 42161:
                return 'eth';
        }

        return '';
    }
}

Final Outcome

After the changes the updated diagram shows how the SOR is more focused and less opinionated:

The plan for the Balancer front-end is to move away from using the SOR directly and use it via the SDK package. The SDK will have the data fetching functionality as serparate services (which can be used independetly for fetching pools, etc) and these will be passed to the SOR when the SDK is instantiated. BUT it’s also possible to use the SOR independendtly as shown in this swapExample.

This was a large and breaking change but with the continued issues with Subgraph and more teams using the SOR/SDK it was a neccessary upgrade. Many thanks to Daniel from the Beethoven team for pushing this through!

Etherscan API

Etherscan is a really useful website but fairly recently I discovered they also have a pretty handy Developer API. I used it to analyse gas a contract was using over a period of time and thought it might be useful to record.

Getting Started

First of all an API key is required and this can be created for free by logging in and going to: https://etherscan.io/myapikey.

I’m going to paste my code with comments which should be easy enough to follow. The API covers a whole lot of other end point such as Logs, Tokens, etc that I didn’t use but the docs are good and worth a scan.

Example

Photo by Markus Spiske on Unsplash

Buidler, Waffle & Ethers

Lately at Balancer we’ve moved from the Truffle development environment to using Buidler, Waffle and Ethers. The main benefit is being able to use console.log in Solidity during debugging – it’s amazing how much of a difference this makes and for this alone the change over is worth it. Here’s some notes I made during the switch over.

Ethers

The ethers.js library aims to be a complete and compact library for interacting with the Ethereum Blockchain and its ecosystem.

Documentation is here: https://docs.ethers.io and this Web3.js vs Ethers.js guide was useful.

The following gist demonstrates some basic usage of Ethers that creates an instance of a deployed contract and then running some calls against it:

Buidler & Waffle

Buidler is described as a ‘task runner’. I think its easiest to see it as a swap for Truffle/Ganache. It has lots of different plugins that make it really useful and its documentation was refreshingly good.

The Quickstart shows you how to install and how to run common tasks. It also uses Waffle for testing. Waffle is a simple smart contract testing library built on top of Ethers.js. Tests in Waffle are written using Mocha alongside with Chai and from my experience everything just worked. The docs are here. And its worth digging in to see some of the useful things it offers such as Chai Matchers which allow you to test things like reverts, events, etc.

Buidler commands I found I used a lot:

  • Run the local Buidler EVM: $ npx buidler node
  • Compile project contracts: $ npx buidler compile
  • Run tests: $ npx buidler test ./test/testfile.ts

Here’s an example test file I used that demonstrates a few useful things:

Static Calls

let poolAddr = await factory.callStatic.newBPool(); – The contract callStatic pretends that a call is not state-changing and returns the result. This does not actually change any state and is free.

Connecting Different Accounts

await _pools[1].connect(newUserSigner).approve(PROXY, MAX); – Using contract connect(signer) calls the contract via the signer specified.

Gas Costs

await proxy.connect(newUserSigner).exitswapExternAmountOut(
            _POOLS[1],
            MKR,
            amountOut,
            startingBptBalance,
            {
              gasPrice: 0
            }
        );

Setting the gasPrice to 0 like above allows me to run the transaction without spending any Eth on it. This was useful when checking Eth balance changes without having to worry about gas costs.

Custom accounts & balances

const config: BuidlerConfig = {
  solc: {
    version: "0.5.12",
    optimizer: {
      enabled: true,
      runs: 200,
    },
  },
  networks: {
    buidlerevm: {
      blockGasLimit: 20000000,
      accounts: [
        { privateKey: '0xPrefixedPrivateKey1', balance: '1000000000000000000000000000000' },
        { privateKey: '0xPrefixedPrivateKey2', balance: '1000000000000000000000000000000' }
      ]
    },
  },
};

I needed the test accounts to have more than the 1000Eth balance set by default. In buidler.config.ts you can add accounts with custom balances like above.

Deploying

Deploying is done using scripts. First I updated my buidler.config.ts with the account/key for Kovan that will be used to deploy (i.e. must have Eth):

const config: BuidlerConfig = {
  solc: {
    version: "0.5.12",
    optimizer: {
      enabled: true,
      runs: 200,
    },
  },
  networks: {
    buidlerevm: {
      blockGasLimit: 20000000,
    }
    kovan: {
      url: `https://kovan.infura.io/v3/${process.env.INFURA}`,
      accounts: [`${process.env.KEY}`]
    }
  },
};

Then I wrote a deploy-script.js:

async function main() {
  // We get the contract to deploy
  const ExchangeProxy = await ethers.getContractFactory("ExchangeProxy");
  const WETH = '0xd0A1E359811322d97991E03f863a0C30C2cF029C';
  const exchangeProxy = await ExchangeProxy.deploy(WETH);

  await exchangeProxy.deployed();

  console.log("Proxy deployed to:", exchangeProxy.address);
}

main()
  .then(() => process.exit(0))
  .catch(error => {
    console.error(error);
    process.exit(1);
  });

Then run this using: npx buidler run --network kovan deploy-script.js

🎉 Console Logging 🎉

One of the holy grails of Solidity development and so easy to setup in this case! There are also Solidity stack traces and error messages but unfortunately there was a bug that caused this not to work for our contracts.

To get this going all you need to do is add: import "@nomiclabs/buidler/console.sol"; at the top of your contract then use console.log. More details on what kind of outputs, etc it supports are here. Lifesaver!

Hope some of this was helpful and you enjoy using it as much as me.

(Photo by Kevin Jarrett on Unsplash)

Setting Up Testing – Typescript with Mocha/Chai

This is more a reminder post incase I ever have to do something like this again so its kind of boring!

For pure Javascript testing with Mocha/Chai:

$ yarn add mocha
$ yarn add chai

In package.json:

"scripts":{
    "test":"mocha"
}

Add a test dir:

$ mkdir test

Add first test file:

var expect  = require('chai').expect;
var request = require('request');

it('Main page content', function(done) {
    request('http://localhost:8080' , function(error, response, body) {
        expect(body).to.equal('Hello World');
        done();
    });
});

Now for Typescript:

$ yarn add typescript
$ yarn add ts-node --dev
$ yarn add @types/chai --dev
$ yarn add @types/mocha --dev

 Replace test script:

"test": "mocha -r ts-node/register test/*.spec.ts"

And make sure test is in root/test/example.spec.ts

import { expect, assert } from 'chai';
import 'mocha';
import { Pool } from '../src/types';
import { smartOrderRouter } from '../src/sor';
import { BigNumber } from '../src/utils/bignumber';
import { getSpotPrice, BONE } from '../src/helpers';

const errorDelta = 10 ** -8;

function calcRelativeDiff(expected: BigNumber, actual: BigNumber): BigNumber {
    return expected
        .minus(actual)
        .div(expected)
        .abs();
}

// These example pools are taken from python-SOR SOR_method_comparison.py
let balancers: Pool[] = [
    {
        id: '0x165021F95EFB42643E9c3d8677c3430795a29806',
        balanceIn: new BigNumber(1.341648768830377422).times(BONE),
        balanceOut: new BigNumber(84.610322835523687996).times(BONE),
        weightIn: new BigNumber(0.6666666666666666),
        weightOut: new BigNumber(0.3333333333333333),
        swapFee: new BigNumber(0.005).times(BONE),
    },
    {
        id: '0x31670617b85451E5E3813E50442Eed3ce3B68d19',
        balanceIn: new BigNumber(14.305796722007608821).times(BONE),
        balanceOut: new BigNumber(376.662367824920653194).times(BONE),
        weightIn: new BigNumber(0.6666666666666666),
        weightOut: new BigNumber(0.3333333333333333),
        swapFee: new BigNumber(0.000001).times(BONE),
    },
];

describe('Two Pool Tests', () => {
    it('should test spot price', () => {
        var sp1 = getSpotPrice(balancers[0]);
        var sp2 = getSpotPrice(balancers[1]);

        // Taken form python-SOR, SOR_method_comparison.py
        var sp1Expected = new BigNumber(7968240028251420);
        var sp2Expected = new BigNumber(18990231371439040);

        var relDif = calcRelativeDiff(sp1Expected, sp1);
        assert.isAtMost(
            relDif.toNumber(),
            errorDelta,
            'Spot Price Balancer 1 Incorrect'
        );

        relDif = calcRelativeDiff(sp2Expected, sp2);
        assert.isAtMost(
            relDif.toNumber(),
            errorDelta,
            'Spot Price Balancer 2 Incorrect'
        );
    });

    it('should test two pool SOR swap amounts', () => {
        var amountIn = new BigNumber(0.7).times(BONE);
        var swaps = smartOrderRouter(
            balancers,
            'swapExactIn',
            amountIn,
            10,
            new BigNumber(0)
        );

        // console.log(swaps[0].amount.div(BONE).toString())
        // console.log(swaps[1].amount.div(BONE).toString())
        assert.equal(swaps.length, 2, 'Should be two swaps for this example.');

        // Taken form python-SOR, SOR_method_comparison.py
        var expectedSwap1 = new BigNumber(635206783664651400);
        var relDif = calcRelativeDiff(expectedSwap1, swaps[0].amount);
        assert.isAtMost(relDif.toNumber(), errorDelta, 'First swap incorrect.');

        var expectedSwap2 = new BigNumber(64793216335348570);
        relDif = calcRelativeDiff(expectedSwap2, swaps[1].amount);
        assert.isAtMost(
            relDif.toNumber(),
            errorDelta,
            'Second swap incorrect.'
        );
    });

    it('should test two pool SOR swap amounts highestEpNotEnough False branch.', () => {
        var amountIn = new BigNumber(400).times(BONE);
        var swaps = smartOrderRouter(
            balancers,
            'swapExactIn',
            amountIn,
            10,
            new BigNumber(0)
        );

        // console.log(swaps[0].amount.div(BONE).toString())
        // console.log(swaps[1].amount.div(BONE).toString())
        assert.equal(swaps.length, 2, 'Should be two swaps for this example.');
        assert.equal(
            swaps[0].pool,
            '0x31670617b85451E5E3813E50442Eed3ce3B68d19',
            'First pool.'
        );
        assert.equal(
            swaps[1].pool,
            '0x165021F95EFB42643E9c3d8677c3430795a29806',
            'Second pool.'
        );

        // Taken form python-SOR, SOR_method_comparison.py with input changed to 400
        var expectedSwap1 = new BigNumber(326222020689680300000);
        var relDif = calcRelativeDiff(expectedSwap1, swaps[0].amount);
        assert.isAtMost(relDif.toNumber(), errorDelta, 'First swap incorrect.');

        var expectedSwap2 = new BigNumber(73777979310319780000);
        relDif = calcRelativeDiff(expectedSwap2, swaps[1].amount);
        assert.isAtMost(
            relDif.toNumber(),
            errorDelta,
            'Second swap incorrect.'
        );
    });

    // Check case mentioned in Discord
});

Photo by René Porter on Unsplash

TypeScript 1 – Getting Going & Migrating

I’m currently working on the Burner Signal project. So far I’ve created the React app that will hopefully be used as the proof of concept.

One problem – so far I’ve done everything in pure Javascript but there’s a strong desire to use TypeScript only.

Oh and another problem – I haven’t developed with TypeScript before! 🤔😂

But…this is a perfect opportunity to learn something new especially as the best way to learn something is to actually build something with it.

So after a bit of reading I do get what the proposed benefits of TypeScript are:

  • Because it uses Types and it transpiles to Javascript the compiler can catch errors – I can definitely see the benefits in this!
  • Using Types is a kind of self documentation
  • IDE integration – dev environments provide lots of TypeScript support which should make it more efficient and faster to develop

I will give it a shot and see if the above is true!

The first thing I need to do is get my current create-react-app which is using pure Javascript migrated to use TypeScript. It was surprisingly easy!

  1. yarn add typescript @types/node @types/react @types/react-dom @types/jest
  2. Change an existing .js file to use .jsx
  3. Restart server – this is important!
  4. That’s it!

Now to learn the basics of TypeScript. For this I’m using the React-TypeScript Cheatsheet and the first suggestion is to get familiar with TypeScript by following 2alitys guide which I’m working through next.

🎉✨🔥 Winner Winner! 🔥✨🎉

Well this is cool – I won the Gitcoin x Aave Hackathon!

My entry was a bot that does arbitrage between two Uniswap exchanges using an Aave Flashloan as the capital for initial trade. The Aave judges were “super impressed” with my work and I got a special mention for the way I overcame a testnet problem by forking UniSwap and customising the code to allow two trading between two exchanges with the same token.

Happy days!