Monthly Archives: December 2009

Web Sockets tutorial with simple Python server

The Landscape: HTML5

HTML5 is an emerging and in-flux client-side standard for developing web applications. It’s really more of a rich client platform specification than just a markup language, including the following slew of new features:

  • canvas for vector graphics
  • video and audio for multimedia
  • local offline storage
  • drag and drop operations
  • Web Socket API for bidirectional client-server communications
  • GeoLocation API
  • standard WYSIWYG HTML editor component
  • Web Workers API for message-passing processes
  • webcam and microphone access
  • 3D graphics rendering engine
  • and more…

A lot of this effort is about wrapping up and building into browsers native support for various (proprietary) technologies already in widespread use on the Web, such as Flash video/webcam/mic and Google Gears offline/drag-and-drop. Others are about cleaning up things for which there currently exist various hacks, and Web Sockets fall into this category.

Introducing Web Sockets

This Chromium blog post contains a nice introduction to Web Sockets:

The Web Sockets API enables web applications to handle bidirectional communications with server-side process in a straightforward way. Developers have been using XMLHttpRequest (“XHR”) for such purposes, but XHR makes developing web applications that communicate back and forth to the server unnecessarily complex. XHR is basically asynchronous HTTP, and because you need to use a tricky technique like long-hanging GET for sending data from the server to the browser, simple tasks rapidly become complex. As opposed to XMLHttpRequest, Web Sockets provide a real bidirectional communication channel in your browser. Once you get a Web Socket connection, you can send data from browser to server by calling a send() method, and receive data from server to browser by an onmessage event handler. A simple example is included below.

In addition to the new Web Sockets API, there is also a new protocol (the “web socket protocol”) that the browser uses to communicate with servers. The protocol is not raw TCP because it needs to provide the browser’s “same-origin” security model. It’s also not HTTP because web socket traffic differers from HTTP’s request-response model. Web socket communications using the new web socket protocol should use less bandwidth because, unlike a series of XHRs and hanging GETs, no headers are exchanged once the single connection has been established. To use this new API and protocol and take advantage of the simpler programming model and more efficient network traffic, you do need a new server implementation to communicate with — but don’t worry. We also developed pywebsocket, which can be used as an Apache extension module, or can even be run as standalone server.

(The mentioned technique of a long-hanging GET is also known as Comet.)

Chrome is presently the only browser that has Web Sockets, and only in the Dev channel releases. Firefox and Safari/WebKit support are under way, according to the implementation status page.

The Web Sockets Protocol

The protocol has the client and server do an HTTP-style handshake, where all text is in UTF-8 and newlines include a carriage return and newline. After this, arbitrary data can be sent back and forth, but delimited in frames, which begin with a 0x00 byte and end with a 0xff byte. Contrast this with the byte stream abstraction presented by raw TCP—having the system hand you whole frames frees the application from having to manually buffer and parse messages out of the stream (which the browser may be able to do more efficiently).

As for how this mixes with browser security policies, the basic gist is that the same-origin policy no longer applies. Requiring the Web Socket to communicate only with the same origin (same host and port as where the HTML/Javascript came from) would be a barrier to deployment because it would require the httpd to additionally speak Web Sockets. (That said, the default port for Web Sockets is in fact port 80.) More generally, this prevents all cross-site communication, which is critical to many classes of applications such as mash-ups and widget dashboards.

But the protocol does require the browser to send the origin information to the server, the server to validate this by echoing the origin, and finally the client to validate that the server echoed this. According to the protocol specs, the response must include the exact same origin, location, and protocol as the request, where:

  • the origin is just the (protocol, host, port) triplet (,
  • the location is the target of the request (ws://, and
  • the protocol is an arbitrary string used to identify the exact application-level protocol expected.

(Note that the origin is different from the Referrer, which includes the full resource path, thus leading to privacy concerns. I hope to write more on the Origin header in a broader context and client-side web security in general soon.)

Example Client and Server

To give you a flavor of how to write a complete end-to-end web application using Web Sockets, the following is a simple client and server application where the server sends two messages down to the client, “hello” and “world.” This example is from my sandbox.

The client-side API for Web Sockets is very simple. The example client just connects to a server on port 9876 and alerts the user of each new message. Just to make this a wholesome HTML5 experience, we’ll write everything in XHTML5 (yes, there exists a stricter, XML flavor of HTML5 for those who preferred XHTML over HTML tag soup):

<!DOCTYPE html>
<html xmlns="">
    <title>Web Socket Example</title>
    <meta charset="UTF-8">
      window.onload = function() {
        var s = new WebSocket("ws://localhost:9876/");
        s.onopen = function(e) { alert("opened"); }
        s.onclose = function(e) { alert("closed"); }
        s.onmessage = function(e) { alert("got: " +; }
      <div id="holder" style="width:600px; height:300px"></div>

Now for the server, which is written in Python. It sends the two messages after a one-second delay before each. Note that the server is hard-coding the response to expect a locally connected client; this is for simplicity and clarity. In particular, it requires that the client is being served from localhost:8888. A real server would parse the request, validate it, and generate an appropriate response.

#!/usr/bin/env python

import socket, threading, time

def handle(s):
  print repr(s.recv(4096))
HTTP/1.1 101 Web Socket Protocol Handshake\r
Upgrade: WebSocket\r
Connection: Upgrade\r
WebSocket-Origin: http://localhost:8888\r
WebSocket-Location: ws://localhost:9876/\r
WebSocket-Protocol: sample
  '''.strip() + '\r\n\r\n')

s = socket.socket()
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('', 9876));
while 1:
  t,_ = s.accept();
  threading.Thread(target = handle, args = (t,)).start()

To run the above, start the Web Socket server (./ and start a web server on port 8888 serving index.html:

./ &
python -m SimpleHTTPServer 8888

Further Exploration

For a more complete application (that’s still reasonably simple), I threw together a real-time X-Y scatter plotting application called Real-Time Plotter. It plots some number of data sources and supports streaming to multiple clients.

The Python server listens for data sources on port 9876. It expects a stream of text, where the first line is the name of the data source and each subsequent line contains a space-separated x-y pair of floating point numbers in the series to be plotted. It listens also on port 9877 for Web Socket clients. A simple data source that issues a random y-value per second can be started from the shell using netcat:

  echo 'my data source'
  while true ; do
    echo "${i}000 $RANDOM"
    sleep 1
} | nc localhost 9876

The client page uses Web Sockets to connect to the server and fetch historical data, as well as start streaming new data. Plotting is done using Flot, a jQuery library for generating decent-looking plots. For throttling when the server is streaming new points quickly, the client only fetches new data (by sending an empty frame) after a complete redraw; the server responds by sending a batch of all new points since the last fetch. (Note: the server’s pump routine currently treats the x values as millisecond timestamps and only issues a single point per second, but this can be easily tweaked/removed.)

Web Sockets can also be used over TLS. This is done by using wss: instead of ws: in the URL, and this defaults to the HTTPS port 443.