Why Websocket ???
HTTP protocol is connection-less and only the client can request information from a server. In any case, a server can contact a client… HTTP is purely half-duplex. Furthermore, a server can answer only one time to a client request.
Some websites or web applications require the server to update client from time to time. There were a few ways to do so:
- the client request the server at a regular interval to check if there is a new information available
- the client send a request to the server and the server answers as soon as he has an information to provide to the client (also known as long time polling)
But those methods have many drawbacks due to HTTP limitation.
So a new protocol has been designed: websockets, which allows a 2 ways communication (full duplex) between a client and a server, over a single TCP connection. Furthermore, websockets re-use the HTTP connection it was initialized on, which means it uses the standard TCP port.
How does websocket work ???
Basically, a websocket start with a HTTP request like the one below:
GET / HTTP/1.1 Upgrade: websocket Connection: Upgrade Sec-WebSocket-Version: 13 Sec-WebSocket-Key: avkFOZvLE0gZTtEyrZPolA== Host: localhost:8080 Sec-WebSocket-Protocol: echo-protocol
The most important part is the “Connection: Upgrade” header which let the client know to the server it wants to change to an other protocol, whose name is provided by “Upgrade: websocket” header.
When a server with websocket capability receive the request above, it would answer a response like below:
HTTP/1.1 101 Switching Protocols Upgrade: websocket Connection: Upgrade Sec-WebSocket-Accept: tD0l5WXr+s0lqKRayF9ABifcpzY= Sec-WebSocket-Protocol: echo-protocol
The most important part is the status code 101 which acknoledge the protocol switch (from HTTP to websocket) as well as the “Connection: Upgrade” and “Upgrade: websocket” headers.
From now, the TCP connection used for the HTTP request/response challenge is used for the websocket: whenever a peer wants to interact with the other peer, it can use the it.
The socket finishes as soon as one peer decides it or the TCP connection is closed.
HAProxy and Websockets
As seen above, there are 2 protocols embedded in websockets:
- HTTP: for the websocket setup
- TCP: websocket data exchange
HAProxy must be able to support websockets on these two protocols without breaking the TCP connection at any time.
There are 2 things to take care of:
- being able to switch a connection from HTTP to TCP without breaking it
- smartly manage timeouts for both protocols at the same time
Fortunately, HAProxy embeds all you need to load-balance properly websockets and can meet the 2 requirements above.
It can even route regular HTTP traffic from websocket traffic to different backends and perform websocket aware health check (setup phase only).
The diagram below shows how things happens and HAProxy timeouts involved in each phase:
During the setup phase, HAProxy can work in HTTP mode, processing layer 7 information. It detects automatically the Connection: Upgrade exchange and is ready to switch to tunnel mode if the upgrade negotiation succeeds. During this phase, there are 3 timeouts involved:
- timeout client: client inactivity
- timeout connect: allowed TCP connection establishment time
- timeout server: allowed time to the server to process the request
If everything goes well, the websocket is established, then HAProxy fails over to tunnel mode, no data is analyzed anymore (and anyway, websocket does not speak HTTP). There is a single timeout invloved:
- timeout tunnel: take precedence over client and server timeout
timeout connect is not used since the TCP connection is already established 🙂
Testing websocket with node.js
node.js is a platform which can host applications. It owns a websocket module we’ll use in the test below.
So basically, I’ll have 2 servers, each one hosting web pages on Apache and an echo application on websocket application hosted by nodejs. High-availability and routing is managed by HAProxy.
In this configuration, the websocket and the web server are on the same application.
HAProxy switches automatically from HTTP to tunnel mode when the client request a websocket.
defaults mode http log global option httplog option http-server-close option dontlognull option redispatch option contstats retries 3 backlog 10000 timeout client 25s timeout connect 5s timeout server 25s # timeout tunnel available in ALOHA 5.5 or HAProxy 1.5-dev10 and higher timeout tunnel 3600s timeout http-keep-alive 1s timeout http-request 15s timeout queue 30s timeout tarpit 60s default-server inter 3s rise 2 fall 3 option forwardfor frontend ft_web bind 192.168.10.3:80 name http maxconn 10000 default_backend bk_web backend bk_web balance roundrobin server websrv1 192.168.10.11:8080 maxconn 10000 weight 10 cookie websrv1 check server websrv2 192.168.10.12:8080 maxconn 10000 weight 10 cookie websrv2 check
The configuration below allows to route requests based on either Host header (if you have a dedicated host for your websocket calls) or Connection and Upgrade header (required to switch to websocket).
In the backend dedicated to websocket, HAProxy validates the setup phase and also ensure the user is requesting a right application name.
HAProxy also performs a websocket health check, sending a Connection upgrade request and expecting a 101 response status code. We can’t go further for now on the health check for now.
Optional: the web server is hosted on Apache, but could be hosted by node.js as well.
defaults mode http log global option httplog option http-server-close option dontlognull option redispatch option contstats retries 3 backlog 10000 timeout client 25s timeout connect 5s timeout server 25s # timeout tunnel available in ALOHA 5.5 or HAProxy 1.5-dev10 and higher timeout tunnel 3600s timeout http-keep-alive 1s timeout http-request 15s timeout queue 30s timeout tarpit 60s default-server inter 3s rise 2 fall 3 option forwardfor frontend ft_web bind 192.168.10.3:80 name http maxconn 60000 ## routing based on Host header acl host_ws hdr_beg(Host) -i ws. use_backend bk_ws if host_ws ## routing based on websocket protocol header acl hdr_connection_upgrade hdr(Connection) -i upgrade acl hdr_upgrade_websocket hdr(Upgrade) -i websocket use_backend bk_ws if hdr_connection_upgrade hdr_upgrade_websocket default_backend bk_web backend bk_web balance roundrobin option httpchk HEAD / server websrv1 192.168.10.11:80 maxconn 100 weight 10 cookie websrv1 check server websrv2 192.168.10.12:80 maxconn 100 weight 10 cookie websrv2 check backend bk_ws balance roundrobin ## websocket protocol validation acl hdr_connection_upgrade hdr(Connection) -i upgrade acl hdr_upgrade_websocket hdr(Upgrade) -i websocket acl hdr_websocket_key hdr_cnt(Sec-WebSocket-Key) eq 1 acl hdr_websocket_version hdr_cnt(Sec-WebSocket-Version) eq 1 acl hdr_host hdr_cnt(Sec-WebSocket-Version) eq 1 http-request deny if ! hdr_connection_upgrade ! hdr_upgrade_websocket ! hdr_w ebsocket_key ! hdr_websocket_version ! hdr_host ## ensure our application protocol name is valid ## (don't forget to update the list each time you publish new applications) acl ws_valid_protocol hdr(Sec-WebSocket-Protocol) echo-protocol http-request deny if ! ws_valid_protocol ## websocket health checking option httpchk GET / HTTP/1.1\r\nHost:\ ws.domain.com\r\nConnection:\ Upgrade \r\nUpgrade:\ websocket\r\nSec-WebSocket-Key:\ haproxy\r\nSec-WebSocket-Version 13\r\nSec-WebSocket-Protocol:\ echo-protocol http-check expect status 101 server websrv1 192.168.10.11:8080 maxconn 30000 weight 10 cookie websrv1 check server websrv2 192.168.10.12:8080 maxconn 30000 weight 10 cookie websrv2 check
Note that HAProxy could also be used to select different Websocket application based on the Sec-WebSocket-Protocol header of the setup phase. (later I’ll write the article about it).
- More information about websockets: http://en.wikipedia.org/wiki/WebSocket/li>
- Websocket RFC: http://tools.ietf.org/html/rfc6455
- HAProxy Technologies
- Aloha load balancer: HAProxy based LB appliance
- HAPEE: HAProxy Enterprise Edition