1 Introduction

1.1 Continuous-time double auction and LOB

Today, most liquid markets – including stocks, futures, and foreign exchange – are electronic, and adopt a continuous-time double auction mechanism using a limit order book (LOB), in which a transaction occurs whenever a buyer and a seller agree on a price(Bouchaud and Bonart 2018). Cryptocurrency exchanges are not exception.

The mechanics of the continuous double auction or LOB trading may be briefly described as follows:

  1. Traders submit limit orders and market orders
  2. Market orders are matched with limit orders and (often) executed to produce trades.
  3. Unmatched amounts of limit orders reside in LOB’s queues until matched with another market order or cancelled by the trader who submitted it.

Market orders are also often called takers (of liquidity) while limit orders sitting in queues are called makers (of liquidity).

Note that not all matched orders are necessarily executed to produce trades. If a market order and a matched limit order are placed by the same trader, an execution would lead to a self-trade. So called self-match or self-trade prevention rules stipulate that in this situation some or both of the matched orders have to be cancelled instead of being executed. See for example CME Globex Self-Match Prevention or Coinbase Markets Trading Rules 2.4 Self-trade prevention.

Thus even if one can figure out from an exchange provided data that a market order has been placed, he can deduce neither whether the trade(s) is going to be produced from the market order nor the amount of the trade(s). In order to reconstruct the dynamics of the countinous-time double auction completely one needs information about all submitted limit and market orders as well as about all trades produced. As we will see below not all exchanges provide this information.

1.2 Available data sets

Most of data sets containing information about orders and trades consist of two separate loosely coupled files: a trades file recording trades and an orders file recording limit and sometimes market order placements, changes and cancellations. Loosely coupled in this context means that records in trades file do not always have clearly identifiable corresponding records in orders file as one would expect since by definition every trade should change some limit order. A matching or coupling procedure is required in order to establish the link between the trades and orders files in order to

  • Distinguish between limit order changes due to cancellation and market order execution
  • Calculate sizes of submitted market orders

and, overall, to achieve the ultimate goal - to perform a complete order book reconstruction at every moment of time.

A recently published book (Abergel 2016) uses the Thomson Reuters Tick History (TRTH) database tells us that

Because one cannot distinguish market orders from cancellations just by observing changes in the limit order book (the “event” file), and since, the time stamps of the “trade” and “event” files are asynchronous, we use a matching procedure to reconstruct the order book events.

The reported matching rate of the above procedure is about 85% for CAC 40 stocks and as a byproduct the procedure outputs the sign of each matched trade, that is whether it is a buyer or a seller initiated trade. Note that TRTH data set does not even provide information about trade direction, it has to be deduced!

The description of similar issues we find in (Hautsch 2004):

A typical problem occurs when trades and quotes are recorded in separate trade and quote databases, like, for example, in the Trade and Quote (TAQ) database released by the NYSE. In this case, it is not directly identifiable whether a quote which has been posted some seconds before a transaction was already valid at the corresponding trade.

Websocket API v2 of the cryptocurrency exchange Bitstamp gives access to the following information for every instrument traded:

  • Live ticker channel - information about trades. Unique ids of participating limit and market order are provided for every trade,
  • Live orders channel - information about limit and market orders (all order creation, change and deletion events are reported).

As we will see later, events in these channels are not always sent in correct order. Sometimes some events appears to be omitted. Time stamps in Live tickers and Live orders channels are not synchronized: market order, changes of limit orders and of market order itself caused by the execution of the market order, trades produced - all may have different time stamps. Substantial amount of orders which appears to be matched are not executed due to self-trade prevention policy of Bitstamp.

Similarly Websocket API version 2.0 of the cryptocurrency exchange Bitfinex have the following channels (for every instrument traded):

  • Trades channel - information about trades. Ids of participating limit and market orders are not provided,
  • Raw book channel - provides information about 100 best bid and 100 ask limit orders. Market orders are not reported at all. A limit order is reported as deleted when it falls beyond 100+ best limit orders and as re-created (i.e. with the same id) if it returns back. What happens to the order after it has fallen beyond 100+ is not known: Bitfinex allows traders to change the price and volume of submitted limit order, so the order can be changed, cancelled or stay unchanged.

As well as at Bitstamp, records in Trades and Raw book channels are not synchronized.

It should be clear from the above that substantial effort is required to reconstruct the true dynamics of order submission, matching and execution.