Author: Nicholas Piano
Published: 2022-07-17
Datafeed Article 239
This article has been digitally signed by Edgecase Datafeed.
This article has been digitally signed by its author.
16 minute read (20 pages)




Executive Summary



In order to deploy any smart contract on the Ethereum blockchain, the following systems must be in place:

  1. Contract-level event emission combined with a subgraph or other means of logging blockchain events. This should incorporate an API to easily access the event logs and any data derived or aggregated from events.
  2. A TypeScript SDK with typed objects reflecting the API provided by the subgraph. This should expose object methods that allow any transaction needed to control the contract to be constructed and dispatched.
  3. A user interface to interact with, create, and manipulate blockchain data related to the application.


This list is in addition to the deployment and testing of the contract itself. When planning a new blockchain application, time and resources should be allotted to these concerns to avoid unnecessary delays.

At a high level, these points also apply to any other non-Ethereum blockchain that supports smart contracts such as Solana or Cardano.




Introduction



You might have been drawn in by the allure of Solidity and its ability to represent a program whose execution is, in reality, spread across multiple transactions encompassing limitless complexity, forming only a small part of a world-spanning computer system promising to re-invent, re-vitalise, reform life on Earth as we know it, but there is a lot more to this riddle.

The problems begin when you try to do something useful. If by "useful" we mean an application that performs a valuable function, such as a bank or an exchange, the Solidity contract itself is insufficient to run a smart contract in the real world. The following should be taken into account when planning to run an application using a smart contract.

There are three main problems: getting data out, getting it in, and displaying it.

The problem of getting data out requires a database to be set up connected to a system watching for blockchain events which are then transformed into updates to the persistent data model. The data can be then be queried as one would query any database, allowing it to be displayed in graphs or tables.

The problem of getting data in requires a transformation from the world of numbers and letters understood by the web, or applications that understand traditional APIs, into the world of Solidity, which only understands very specific formats compatible with the Ethereum blockchain. Once these transformations are in place, they can make it very simple to re-use and build on the underlying contracts.

Once there are ways for data to get into and out of the contracts, a user interface can make use of both of them. The transformation of data going in can lie between the press of a button and the execution of a Solidity function. The data coming out will pass through the database before it is queried by the interface and displayed in a useful form.

For each problem, I will discuss the general case along with a specific solution that I found.




Problem #1: Getting data out



Solidity is designed to provide an abstraction to the movement of data across transactions, but once the data is in place, it is inefficient and expensive to interact with it. Writing more than reading, but nothing is free.

If the smart contract is building up a system of interconnected data, e.g. user balances in a bank, any user of the system will need to quickly answer questions about it, such as their current balance or balance history. The smart contract is not designed to answer these types of questions, so another system is needed to fill this role.

The answer lies with smart contract events. Each function call from within the contract can emit events that allow changes in data to be tracked over time. An example might look like:

function updateBalance() {
  ...
  emit UpdateBalance(balance);
}


Now, each time this function is called, I can build up a history of a balance based on the events:

tx 111 - balance: 3
tx 116 - balance: 5
tx 131 - balance: 1
tx 145 - balance: 7


Once this in place, you need something to watch for events and update a data model that represents the system. This is done by parsing each block that gets mined and extracting the events embedded within. Unless you want to run a node yourself and parse each block using custom software, you will need to make use of a service built for this purpose. Among them are Infura, Alchemy, and The Graph, which offer a range of different utilities for logging blockchain data.




The Graph



The Graph is the simplest example. The data model resulting from data extracted from events is referred to as a "subgraph" and is intended to be queried like any database. Using the balance example from above, our intended data model might look something like this:

Token
  address

User
  address

Balance
  user
  token
  value


This model allows for a user to have balances in any number of tokens deployed on the blockchain. The subgraph requires a configuration file that describes how to extract data from the blockchain. You can either listen to events from a contract with a specific address (Data Sources in subgraph terms), or instances of contract code that have not yet been deployed (known as Templates). Suppose the events are emitted from the following contract:

contract Bank {

  event UpdateBalance(indexed address, indexed address, int256);

  ...

  function updateBalance(owner, token, balance) {
    ...
    emit UpdateBalance(owner, token, balance);
  }

  ...

}


The subgraph configuration (
subgraph.yaml
) would need to include the following:

schema:
  file: ./schema.graphql
dataSources:
  - kind: ethereum/contract
    name: Bank
    network: mainnet
    source:
      address: "0x9fE46736679d2D9ddfF0992F2272dE9f3c7fa6e0"
      abi: Bank
    mapping:
      kind: ethereum/events
      apiVersion: 0.0.5
      language: wasm/assemblyscript
      file: ./src/mappings/bank/index.ts
      entities:
        - User
        - Token
        - Balance
        - Deposit
        - Withdrawal
      abis:
        - name: Bank
          file: ./artifacts/Bank.json
      eventHandlers:
        - event: UpdateBalance(indexed address,indexed address,int256)
          handler: handleUpdateBalance


Some configuration options have been omitted, such as the repository URL, and there are several things to notice here, but it's helpful to back up and refocus on the objective of this exercise: extracting data from the blockchain and using it to build a data model that allows complex questions to be answered efficiently. This configuration file lies at the intersection of several pieces of infrastructure that make that happen.

  1. The contract ABIs - ABIs are the compiled form of a Solidity contract in preparation for deployment to the blockchain
  2. The database - a database needs to hold and associate pieces of data gathered from events
  3. The GraphQL server - GraphQL is not required, but provides a consistent way to access data for clients


The ABIs are indicated by the options
dataSources.source.abi
and
dataSources.mapping.abis
. These specify how to find the results of the Solidity compilation.

The data model is defined in the
schema.file
option and referenced in
dataSources.mapping.entities
. The data model does not need to correspond one-to-one to entities defined in the contracts. They can take any form that fulfils their purpose. For example, a data model could register a
Deposit
with the balance difference if it is positive and a
Withdrawal
otherwise. The original contract has no knowledge of either of these.

The final piece of the puzzle is the handlers, which are defined in
dataSources.mapping.eventHandlers
. The TypeScript file containing the mapping code is defined in
dataSources.mapping.file
.




GraphQL schema



The schema is defined independently of the contracts and can take any shape. An example schema could look like:

# Objects
type Token @entity {
  # token address
  id: ID!

  name: String!
  decimals: BigInt!
}

type User @entity {
  # user address
  id: ID!
  updateCount: BigInt!

  deposits: [Deposit!]! @derivedFrom(field: "user")
  withdrawals: [Withdrawal!]! @derivedFrom(field: "user")
}

type Balance @entity {
  token: Token!
  user: User!

  value: BigInt!
}

# Event tracking
type Transaction @entity {
  # transaction hash
  id: ID!
  # block transaction was included in
  blockNumber: BigInt!
  # timestamp transaction was confirmed
  timestamp: BigInt!
  # gas used during transaction execution
  gasPrice: BigInt!

  # derived values
  deposits: [Deposit!]! @derivedFrom(field: "transaction")
  withdrawals: [Withdrawal!]! @derivedFrom(field: "transaction")
}

type Deposit @entity {
  # format: transaction hash + "#" + index in deposits Transaction array
  id: ID!

  transaction: Transaction!
  user: User!
  value: BigInt!
}

type Withdrawal @entity {
  # format: transaction hash + "#" + index in withdrawals Transaction array
  id: ID!

  transaction: Transaction!
  user: User!
  value: BigInt!
}


Note that the model for
Deposit
is very similar to
Withdrawal
so they could be combined into one. This is not necessarily the most efficient data model!

TypeScript code can be generated from this schema because it is already strongly typed.

IMPORTANT: Because the
wasm/assemblyscript
option is selected, the TypeScript code must be compatible with the WebAssembly compiler. This can include unfamiliar types such as
i32
for integers, which are not present in vanilla JavaScript.




Event handlers



The purpose of an event handler is to transform the incoming data from an event into data that will be fed into the data model. This can incorporate existing data that is referenced in the event, such as the
owner
address in this example.

IMPORTANT: The Graph requires the handler function to be a JavaScript
function
object (not an anonymous arrow function).

The handler can be defined as follows:

// ./src/mappings/bank/handleUpdateBalance.ts
function handleUpdateBalance(event: UpdateBalanceEvent) {
  const token = getOrCreateToken(event.params.token.toString());
  const user = getOrCreateUser(event.params.user.toString());
  const balance = getOrCreateBalance(token, user);
  const newBalanceValue = BigInt.fromI32(event.params.balance);
  const balanceDifference = newBalanceValue.minus(balance.value);
  const transaction = getOrCreateTransaction(
    event.transaction.hash.toHexString()
  );

  user.updateCount = user.updateCount.plus(1);
  user.save();

  const updateId = `${transaction.id.toString()}#${user.updateCount.toString()}`;

  if (balanceDifference.gt(BigInt.fromI32(0))) {
    const deposit = new Deposit(updateId);
    deposit.token = token;
    deposit.user = user;
    deposit.value = balanceDifference;
    deposit.save();
  } else {
    const withdrawal = new Withdrawal(updateId);
    withdrawal.token = token;
    withdrawal.user = user;
    withdrawal.value = balanceDifference;
    withdrawal.save();
  }

  balance.value = newBalanceValue;
  balance.save();
}

export default handleUpdateBalance;


The
index.ts
file then exports each function:

// ./src/mappings/bank/index.ts
export { default as handleUpdateBalance } from "./handleUpdateBalance";


Now the handlers will be picked up correctly by the configuration file.




A running subgraph



When running, a subgraph acts as a server that allows access to its database using GraphQL. From the perspective of any client, it is simply a database that does not accept writes. In the background, it is listening to an Ethereum node to receive block data.

With this in place, we now have a simple interface from the blockchain to the outside world. An SDK can be created that represents entities as objects in JavaScript, for example, allowing the next phase to be started: a way to get data into the system and onto the blockchain.




Problem #2: Getting data in



Every interaction that adds or changes blockchain data comes in the form of a transaction. This requires authentication from the sender of the transaction in the form of signing, and requires that the target contract accept the sender as an authorised agent.

Technically, a transaction can be written manually. It is simply a string of bytes that adheres to a set of rules. In reality though, it is useful to wrap the construction process in layers of utilities that abstract away the repeatable parts, such as formatting or signing. At the top level, a utility can be used to construct the specific types of transactions needed to interact with our contract. This utility can be written into an SDK for easy re-use. An SDK then performs two functions:

  1. Accept blockchain data from the subgraph and use it to assemble formal objects
  2. Provide objects with methods that construct and broadcast transactions by combining object attributes with method arguments


To represent our data, we could use objects like these:

class Bank {
  constructor({ address }) {
    this.address = address;
  }
}

class Account {
  constructor({ bank, token, address, balance }) {
    this.bank = bank;
    this.token = token;
    this.address = address;
    this.balance = balance;
  }

  async updateBalance(newBalance) {
    // make call to blockchain using attributes
  }
}


The implementation of the
updateBalance
method will be discussed in the next section, but the general idea allows the function to be called in the following manner:

const bank = new Bank({ address: BANK_ADDRESS });
const account = new Account({
  bank,
  token: TOKEN_ADDRESS,
  address: ACCOUNT_ADDRESS,
  balance: ACCOUNT_BALANCE
});

await account.updateBalance(NEW_BALANCE);


Each of these constants could be loaded from the subgraph API or entered using a form in the UI.




Implementing a contract call



The
updateBalance
method can built using
Contract
object from the
ethers
package. This takes a contract ABI and wraps it to provide access to its functions. Further, the
typechain
package wraps this functionality again to represent the specific contract as a typed object in TypeScript. The method could take the form:

import { BankContract } from './typechain'; // generated typed contract wrappers

...

async updateBalance(newBalance) {
  const signer = // a ethers.JsonRpcSigner object, such as from metamask
  const bankConnection = BankContract.connect(BANK_ADDRESS, signer);

  // Remember that the contract function takes (owner, token, balance)
  const updateBalanceTransaction = await bankConnection.updateBalance(
    this.address, // the account, or "owner" address
    this.token,
    newBalance,
  );

  // At this point, the `updateBalanceTransaction` object is of type `ethers.ContractTransaction`.
  // When the promise completes, it signifies that the transaction has been
  // broadcast, but not yet accepted and mined. This is completed in the following step:

  const updateBalanceTransactionReceipt = await updateBalanceTransaction.wait(
    // optionally, takes the number of blocks to wait as an argument
  );

  // The `updateBalanceTransactionReceipt` represents a mined transaction, including attributes
  // such as `.hash`.

  return updateBalanceTransactionReceipt;
}


Ideally, the stages of the transaction would be captured as state changes that could be reflected in the UI, as discussed in a previous article: Don't use hooks to load data.

A saga for loading this might look like:

function* updateBalanceSaga(action) {
  const { address, token, balance } = action.payload;
  const signer = // a ethers.JsonRpcSigner object, such as from metamask
  const bankConnection = BankContract.connect(BANK_ADDRESS, signer);

  const updateBalanceTransaction = yield call([bankConnection, 'updateBalance'], address, token, balance);

  // This captures the dispatch state change
  yield put(actions.transactionDispatched(updateBalanceTransaction));

  const updateBalanceTransactionReceipt = await call([updateBalanceTransaction, 'wait'], blocksToWait);

  // Capture mined transaction
  yield put(actions.transactionConfirmed(updateBalanceTransactionReceipt));
}





A complete SDK



In summary, an SDK must be written to receive data from the subgraph API and construct transactions compatible with the Solidity contract. It should be a one-stop shop for interacting with the blockchain data.

However, it should also be noted that multiple completely valid SDKs can be written for the same contract. It depends on the subgraph configuration and the aspect of the contract we want to expose. For example, one SDK might focus on account management for the bank, making it easy to make updates to the contract. Another SDK might be written to aggregate data about all accounts in the bank without providing functionality useful for a single account.




Problem #3: Displaying data



The last part of the picture is the display of the data itself. A user interface allows the user to see and manipulate the data represented by the contract. This could be as simple as providing a list of deposits and withdrawals to providing detailed analysis of a user's account via graphs or other visualisations.

Data output (from the contract) is already handled by the subgraph, which can be easily queried using a system such as Apollo GraphQL or a series of sagas using
redux-saga
.

Data input is performed using the SDK object methods. The SDK (or any SDK) can be deployed as an NPM package that can be easily imported into the UI package.

An essential piece of infrastructure needed to tie this together will be support for a browser wallet such as Metamask or Wallet Connect. Data from this will feed into the methods called by the SDK and allow transactions to be signed by the user.

In short, the following pieces are needed to build a UI for the smart contract:

  1. A means by which to make calls to the subgraph API, e.g. Apollo GraphQL
  2. An SDK NPM package that can be imported to easily represent blockchain data as objects
  3. A connection to a browser wallet plugin such as Metamask


With these tools, any UI can be created to interact with the contracts.




Conclusion



Running a smart contract is more complicated than it might appear, but by combining a few other systems that act as gateways for data coming in and out, it can allow the power of the blockchain to be fully exploited. In the same manner as a car engine requires both wheels and a steering wheel to become useful, a smart contract needs these auxiliary systems. It doesn't matter how powerful an engine is if it remains motionless on the ground. Similarly, the blockchain is nothing if it cannot be accessed easily.