Configure syslog-ng to log readable HTTP URL from HAProxy

This tips is provided by Exosec.
Exosec provides a very good monitoring product called POM, based on Nagios with very strong value added such as very simple administration, application monitoring, etc…
For some of their project, they use either HAProxy or the ALOHA Load-Balancer (heh, what else???) and they export log entries into syslog-ng for storage and later analysis.

HAProxy’s log

HAProxy’s logs are very powerfull since they provide many information about the request and the status of the platform at the moment of the request.
For a readable HAProxy’s log description, please consult the ALOHA memo dedicated to HAProxy’s HTTP log line description.

One of the weakness of the log line is that it logs only the path and the query string of each URL. No server name neither protocol information.
Well, HAProxy allows us to log the Host header, which is fine and there is a tild ‘~’ after the frontend name when the connection is made over SSL.

Using syslog-ng flexible configuration, we can re-order things to make haproxy log the URL exactly like it was sent by the client, like:


HAProxy configuration

Note: this is a very minimalistic configuration, not recommended in production 🙂

 log local2

frontend ft_http
 bind ssl crt /etc/haproxy/haproxy.pem
 option http-server-close
 mode http
 log global
 option httplog
 # Mandatory to build the URL:
 capture request header Host       len 32
 # Optional, just for statistics:
 capture request header User-Agent len 200
 default_backend bk_http

backend bk_http
 option http-server-close
 mode http
 log global
 option httplog

 server srv1

Syslog-ng configuration

The configuration below will reproduce an HAProxy log line, but will replace the URL part by something more readable.

Note: if you capture a different number of HTTP headers in HAProxy (current example contains 2 captured headers), you may have to update the parser p_haproxy_headers_req and the destination d_haproxy_full.

source s_loopback { syslog(ip( port(514) transport("udp")); };

destination d_haproxy_full {
          group(adm) perm(0640) dir_perm(0750) template_escape(no)

filter f_haproxy { program("haproxy"); };
filter f_frontend_ssl { match("~ "); };

rewrite r_set_frontend_protocol {
  set("http", value("HAPROXY.FRONTEND_PROTOCOL") condition(filter(f_haproxy)));
  set("https", value("HAPROXY.FRONTEND_PROTOCOL") condition(filter(f_frontend_ssl)));

parser p_haproxy {
  csv-parser(columns("HAPROXY.CLIENT_IPPORT", "HAPROXY.DATE",
                     "HAPROXY.FRONTEND", "HAPROXY.BACKEND",
                     "HAPROXY.TIME", "HAPROXY.STATUS_CODE",
                     "HAPROXY.BYTES_READ", "HAPROXY.COOKIE_REQ",
                     "HAPROXY.RUN_STATE", "HAPROXY.QUEUE_STATE",
                     "HAPROXY.HEADERS_REQ", "HAPROXY.REQUEST")
             delimiters(" ")

parser p_haproxy_request {
  csv-parser(columns("HAPROXY.METHOD", "HAPROXY.URL",
             delimiters(" ")

parser p_haproxy_headers_req {
  csv-parser(columns("HAPROXY.HOST", "HAPROXY.USER_AGENT")

log {

This configuration is downloadable from HAProxy Technologies github:

And the result would look like below at the end of the logged line:

[...] "GET http://test.domain.tld/blah HTTP/1.1"
[...] "GET https://test.domain.tld/blah HTTP/1.1"


About Baptiste Assmann

Aloha Product Manager HAProxy consultant
This entry was posted in HAProxy, optimization and tagged , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s