Most exchanges provide a public websocket service that sends real-time market data to you. You can use this free service to automate trading or perform quick calculations that are not easy to perform on the exchange’s charts. This data arrives in massive amounts, sometimes making real-time processing of the data difficult.

In this tutorial, I will show you how to subscribe to a websocket on Binance, and then we will do some cool things with the stream.

What we will accomplish:

  1. Sign up for Binance and get your API key
  2. Set up a websocket connection to Binance
  3. Process the data into a useful form
  4. Perform some calculations on realtime data
  5. Look at a better way of accomplishing the same thing

 

Sign up for Binance & Get API Keys

 

The first step is to get yourself signed up on binance.

 

Once you have signed up, visit your account and click API Setting

 

Then, go ahead and name your API key. We’ll call it binance-websocket-tutorial for now. Because that is what this is, afterall.

 

After you verify, you will see a screen that has your API key and API Secret. You wont be able to see your secret key again, so make note of it.

 

Set up a Websocket Connection to Binance

 

Sammchardy on github has provided an amazing library to interface with Binance’s api. First you’re going to have to install the python-binance wrapper from github like so:

pip3 install python-binance

Sockets are handled through the BinanceSocketManager. It can handle multiple socket connections. When creating a socket connection, a callback function is passed which receives the messages. Lets jump right into an example. We are going to listen to the websocket for the symbol ETHBTC. We want to see every trades as it occurs, and we can do so using Binance’s trade socket. Use the following code sample:

 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
import time
from binance.client import Client # Import the Binance Client
from binance.websockets import BinanceSocketManager # Import the Binance Socket Manager

# Although fine for tutorial purposes, your API Keys should never be placed directly in the script like below. 
# You should use a config file (cfg or yaml) to store them and reference when needed.
PUBLIC = '<YOUR-PUBLIC-KEY>'
SECRET = '<YOUR-SECRET-KEY>'

# Instantiate a Client 
client = Client(api_key=PUBLIC, api_secret=SECRET)

# Instantiate a BinanceSocketManager, passing in the client that you instantiated
bm = BinanceSocketManager(client)

# This is our callback function. For now, it just prints messages as they come.
def handle_message(msg):
    print(msg)

# Start trade socket with 'ETHBTC' and use handle_message to.. handle the message.
conn_key = bm.start_trade_socket('ETHBTC', handle_message)
# then start the socket manager
bm.start()

# let some data flow..
time.sleep(10)

# stop the socket manager
bm.stop_socket(conn_key)

Output: 

1
2
3
{'e': 'trade', 'E': 1529623973721, 's': 'ETHBTC', 't': 69713886, 'p': '0.07836000', 'q': '0.03000000', 'b': 169884767, 'a': 169884759, 'T': 1529623973718, 'm': False, 'M': True}
{'e': 'trade', 'E': 1529623973721, 's': 'ETHBTC', 't': 69713887, 'p': '0.07836100', 'q': '0.03600000', 'b': 169884767, 'a': 169884580, 'T': 1529623973718, 'm': False, 'M': True}
{'e': 'trade', 'E': 1529623974326, 's': 'ETHBTC', 't': 69713888, 'p': '0.07830500', 'q': '0.15600000', 'b': 169884761, 'a': 169884769, 'T': 1529623974324, 'm': True, 'M': True}

Congratulations. You are new streaming cryptocurrency data!

 

Make the Data Useful

 

As you saw earlier, we were streaming real trades from Binance. But, we didn’t get the most useful data printing out. That is because our handle_message function was fairly simple. Let’s work on it a bit to create something more useful.

The trade socket returns a dictionary as specified in Binance’s official api documentation for every trade that occurs. We expect the payload to look like this:

{
  "e": "trade",     // Event type
  "E": 123456789,   // Event time
  "s": "BNBBTC",    // Symbol
  "t": 12345,       // Trade ID
  "p": "0.001",     // Price
  "q": "100",       // Quantity
  "b": 88,          // Buyer order Id
  "a": 50,          // Seller order Id
  "T": 123456785,   // Trade time
  "m": true,        // Is the buyer the market maker?
  "M": true         // Ignore.
}

In the event of an error, the BinanceSocketManager tries reconnecting a maximum of 5 times. If it fails to do so, we should expect a message like this:

{
    'e': 'error',
    'm': 'Max reconnect retries reached'
}

With all of that in mind, let’s create a smarter callback function that will define what to do when we get a trade message. We will call it handle_message.

 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
def handle_message(msg):
    
    # If the message is an error, print the error
    if msg['e'] == 'error':    
        print(msg['m'])
    
    # If the message is a trade: print time, symbol, price, and quantity
    else:
        print("Time: {} Symbol: {} Price: {} Quantity: {} ".format(msg['T'],
                                                                   msg['s'],
                                                                   msg['p'],
                                                                   msg['q']))

 

We will again specify that we want to listen to ETHBTC, and we want handle_message to deal with the trades as they come. Lets try this again.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
import time
from binance.client import Client 
from binance.websockets import BinanceSocketManager

PUBLIC = '<YOUR-PUBLIC-KEY>'
SECRET = '<YOUR-SECRET-KEY>'

client = Client(api_key=PUBLIC, api_secret=SECRET)
bm = BinanceSocketManager(client)

conn_key = bm.start_trade_socket('ETHBTC', handle_message)

bm.start()

time.sleep(10) # let some data flow..

bm.stop_socket(conn_key)

Output:

1
2
3
4
5
6
7
8
9
Time: 1529624001948 Symbol: ETHBTC Price: 0.07829500 Quantity: 0.00100000 
Time: 1529624001948 Symbol: ETHBTC Price: 0.07829200 Quantity: 0.03800000 
Time: 1529624001948 Symbol: ETHBTC Price: 0.07829100 Quantity: 0.01600000 
Time: 1529624002093 Symbol: ETHBTC Price: 0.07829100 Quantity: 0.04600000 
Time: 1529624002363 Symbol: ETHBTC Price: 0.07834700 Quantity: 0.16700000 
Time: 1529624004165 Symbol: ETHBTC Price: 0.07829500 Quantity: 0.10100000 
Time: 1529624004184 Symbol: ETHBTC Price: 0.07829200 Quantity: 0.07600000 
Time: 1529624004217 Symbol: ETHBTC Price: 0.07829200 Quantity: 1.28800000 
Time: 1529624004292 Symbol: ETHBTC Price: 0.07829200 Quantity: 0.56800000

 

Perform Some Calculations on Real Time Data

We can make our handle_message even smarter. Let’s say we were interested in how many bitcoins were exchanging hands per ETHBTC trade. We need to do a few things:

  • Each trade has a price and a quantity. We can multiply that to get the amount of bitcoin traded.
  • Since humans don’t respond with “1529616742353 milliseconds since Thursday, 1 January 1970” when asked for the time, we’ll make the time format human readable.
  • It would also be useful to know if the trade was a buy or a sell.

First lets determine how many bitcoins were exchanged per ETHBTC trade:

 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
def handle_message(msg):
    
    if msg['e'] == 'error':    
        print(msg['m'])
        
    else:
        # Bitcoins exchanged, is equal to price times quantity
        bitcoins_exchanged = msg['p'] * msg['q']
        
        # Print this amount
        print("Time: {} Symbol: {} Price: {} Quantity: {} BTC Exchanged: {}".format(msg['T'], 
                                                                                    msg['s'], 
                                                                                    msg['p'], 
                                                                                    msg['q'],
                                                                                   bitcoins_exchanged))

 

If we run this again…

 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
import time
from binance.client import Client 
from binance.websockets import BinanceSocketManager

PUBLIC = '<YOUR-PUBLIC-KEY>'
SECRET = '<YOUR-SECRET-KEY>'

client = Client(api_key=PUBLIC, api_secret=SECRET)
bm = BinanceSocketManager(client)

conn_key = bm.start_trade_socket('ETHBTC', handle_message)

bm.start()

time.sleep(10) # let some data flow..

bm.stop_socket(conn_key)

 

We run into the following error: builtins.TypeError: can't multiply sequence by non-int of type 'str'

This is a funny error. We would expect price and quantity to come back as numbers, but for some reason they are actually coming to us from BinanceSocketManager as strings. If you look at schema of the dictionary that is returned, you can see for yourself:

"p": "0.001", // Price

"q": "100", // Quantity

I don’t know whose decision that was… but, it’s a good thing we caught that. Let’s try again by turning price and quantity into floats.

 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
def handle_message(msg):
    
    if msg['e'] == 'error':    
        print(msg['m'])
        
    else:
        # Bitcoins exchanged - This time converting the strings to floats.
        bitcoins_exchanged = float(msg['p']) * float(msg['q'])
        
        # Print this amount
        print("Time: {} - Symbol: {} - Price: {} - Quantity: {} BTC Quantity: {}".format(msg['T'], 
                                                                                    msg['s'], 
                                                                                    msg['p'], 
                                                                                    msg['q'],
                                                                                   bitcoins_exchanged))

Output:

1
2
3
4
5
6
7
Time: 1529624811880 - Symbol: ETHBTC - Price: 0.07837300 - Quantity: 4.65200000 BTC Quantity: 0.3645911
Time: 1529624812169 - Symbol: ETHBTC - Price: 0.07833700 - Quantity: 0.10000000 BTC Quantity: 0.0078337
Time: 1529624814290 - Symbol: ETHBTC - Price: 0.07833700 - Quantity: 0.90100000 BTC Quantity: 0.0705816
Time: 1529624814386 - Symbol: ETHBTC - Price: 0.07837400 - Quantity: 0.12700000 BTC Quantity: 0.0099534
Time: 1529624814451 - Symbol: ETHBTC - Price: 0.07833700 - Quantity: 0.05000000 BTC Quantity: 0.0039168
Time: 1529624815140 - Symbol: ETHBTC - Price: 0.07833700 - Quantity: 0.00300000 BTC Quantity: 0.0002350
Time: 1529624815481 - Symbol: ETHBTC - Price: 0.07833700 - Quantity: 0.03000000 BTC Quantity: 0.0023501

 

Now lets make that time a little bit prettier:

 

1
2
3
4
5
6
7
8
9
# Binance returns a UTC timestamp in milliseconds. 
timestamp = 1529618149064
# Convert this timestamp into seconds by dividing by 1000
timestamp = timestamp / 1000
# Use the datetime library to convert this into a datetime
from datetime import datetime
new_time = datetime.fromtimestamp(timestamp)
# Use strftime to make it readable
new_time.strftime('%Y-%m-%d %H:%M:%S')

Output:

1
'2018-06-21 14:55:49'

 

Lets find out if this was a buy or sell event:

 

In our dictionary, we are given a boolean if the buyer is the market maker.

"m": true, // Is the buyer the market maker?

As a refresher, the market maker adds liquidity to the market. If the buyer is a market maker, it means that someone has bought the asset from at his or her price. Therefore, we can classify this as a ‘sell’ event.

 

All Together

 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
from binance.client import Client
from binance.websockets import BinanceSocketManager 
from datetime import datetime
import time
# Although fine for tutorial purposes, your API Keys should never be placed directly in the script like below. 
# You should use a config file (cfg or yaml) to store them and reference when needed.
PUBLIC = '<YOUR-PUBLIC-KEY>'
SECRET = '<YOUR-SECRET-KEY>'

client = Client(api_key=PUBLIC, api_secret=SECRET)
bm = BinanceSocketManager(client)

def handle_message(msg):
    
    if msg['e'] == 'error':    
        print(msg['m'])
        
    else:
        # Bitcoins exchanged - This time converting the strings to floats.
        bitcoins_exchanged = float(msg['p']) * float(msg['q'])
        
        # Make time pretty
        timestamp = msg['T'] / 1000
        timestamp = datetime.fromtimestamp(timestamp).strftime('%Y-%m-%d %H:%M:%S')
        
        # Buy or sell?
        if msg['m'] == True:
            event_side = 'SELL'
        else:
            event_side = 'BUY '
        
        # Print this amount
        print("{} - {} - {} - Price: {} - Qty: {} BTC Qty: {}".format(timestamp,
                                                                       event_side,
                                                                       msg['s'],
                                                                       msg['p'],
                                                                       msg['q'],
                                                                       bitcoins_exchanged))

conn_key = bm.start_trade_socket('ETHBTC', handle_message)

bm.start()

time.sleep(5) # let some data flow..

bm.stop_socket(conn_key)

Output:

1
2
3
2018-06-21 16:48:20 - SELL - ETHBTC - Price: 0.07833300 - Qty: 0.01900000 BTC Qty: 0.001488327
2018-06-21 16:48:20 - SELL - ETHBTC - Price: 0.07832600 - Qty: 0.04200000 BTC Qty: 0.003289692
2018-06-21 16:48:23 - SELL - ETHBTC - Price: 0.07832700 - Qty: 0.06200000 BTC Qty: 0.004856274

 

A Better Way

 

What we did above was pretty cool. We subscribed to a stream of realtime trade data from binance, made it easy to read, and performed a simple calculation on it. But there was a lot of work that we had to do to get there. What if we had to do this for every exchange? A lot of time evaluating the market would be spent getting stuck on issues like dealing with numbers as strings, or cryptic ‘is buyer market maker’ type fields. Every exchange is different too.

We also need to keep in mind that we only start getting new trades. We don’t even have access to historical data yet for binance.

 

LiveDataFrame

 

The good news is, LiveDataFrame has taken care of that for you. It provides access to every coin on Binance (and other exchanges too!), with realtime updates of OHLC data with 5 second granularity. It also provides up to 4 hours of historical data for every coin!

Lets see how easy it is to start with LiveDataFrame. First, sign up and get your api keys.

Then pip install livedataframe to install the client and use the code below:

 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
from livedataframe import LiveExchange, ExchangeInfo

# Lets get a list of all symbols trading on binance
all_symbols_binance = ExchangeInfo.list_symbols_for('binance')

livedf = LiveExchange(
    public_key='<YOUR-PUBLIC-KEY>', # Enter your public key that was emailed
    secret_key='<YOUR-PRIVATE-KEY>', # Enter your secret key that was emailed
    exchange = 'binance', # Enter your exchange
    symbols = all_symbols_binance, # Enter a dict of symbols. How about lets get them all?
    lookback_period = '1H')

livedf.start()

Output:

1
2
Seeding With Data... This Could Take a Few Minutes.
TOTAL TIME TO LOAD: 0:01:02.963685

 

You can access a symbol that is trading on binance using the symbols dictionary. What you get is a DataFrame that is always up to date with the current OHLC data on binance. Not bad for 15 lines of code.

 

1
livedf.symbols['ETHBTC'].tail()

 

Lets look at last price and last trade quantity:

 

1
2
#      access 'ETHBTC'    'last price'  'last trade qty'    '-1 for most recent value'
livedf.symbols['ETHBTC'][['last_price', 'last_trade_qty']].iloc[[-1]]

Output:

last_price last_trade_qty
2018-06-22 06:50:24+00:00 0.078322 0.055

 

Very easy to read isn’t it? Lets do some magic and watch this in real time:

 

1
2
3
4
5
6
7
8
import time
from IPython.display import clear_output # To clear the output.
  
while True:
    try: 
        clear_output(wait=True)
        print(livedf.symbols['ETHBTC'][['last_price', 'last_trade_qty']].iloc[[-1]])
        time.sleep(5)

Output:

1
2
.                          last_price  last_trade_qty
2018-06-22 06:52:19+00:00    0.078287           0.062

 

More:

 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
import time
from IPython.display import clear_output # To clear the output.

while True:
    clear_output(wait=True)
    print(livedf.symbols['ETHBTC'][['symbol', 'last_price', 'last_trade_qty']].iloc[[-1]])
    print('\n')
    print(livedf.symbols['LTCBTC'][['symbol', 'last_price', 'last_trade_qty']].iloc[[-1]])
    time.sleep(5)

Output:

1
2
3
4
5
6
.                          symbol  last_price  last_trade_qty
2018-06-22 06:52:49+00:00  ETHBTC    0.078365           0.544


                           symbol  last_price  last_trade_qty
2018-06-22 06:52:49+00:00  LTCBTC    0.014386             9.4

Check out our documentation to see what else you can do with LiveDataFrame!

 

DISCLAIMER: The above references an opinion and is for information purposes only. It is not intended to be investment advice. Seek a duly licensed professional for investment advice.