Load Balance HA MySQL with HAProxy.
HAProxy or High Availability Proxy is a high performance open-source load balancer created to work (load balance) over TCP/HTTP. This is a Software that has several use cases you can chose, but it’s great for a MySQL load balancer. The company who makes HAProxy also develops many more products paid and free. Now for this lab we will use the community version as load balancer software on our servers. The key theme for this post will be HAProxy Docker to Load Balance MySQL. In other words a lab of HAProxy, Docker and MySQL.
In general Load Balancers power the uptime of organizations with large infrastructures and enormous traffic demands by giving them the flexibility and confidence to deliver websites and applications with high availability, performance and security at any scale in any environment. Many companies use load balancers in their product delivery process. You can check this article to configure a Master-Slave on MySQL.

In this post, we will create a master replication master on MySQL and make MySQL load balancer replication with HAProxy. Load balance is useful if we want to create a reliable database with multiple MySQL services on multiple instances or servers. For this purpose, I have used docker to create a load balance with HAProxy and MySQL. However, you can also deploy it on an AWS instance or server without running docker.
Okay, let’s get started with creating a Master-Master replication with load balance on HAProxy using MySQL.
Note: I am using Ubuntu Server 18.04 LTS when creating this article.
How To Load Balance with MySQL: Editing The Docker-Compose yml File
If we create MySQL replication with HAProxy load balancer to load Balance using docker containers. The easiest way or you could even say best way for a lab is to make it is to use Docker Compose. This way you configure docker to create multiple containers using a dot yml (yaml) configuration file.
Docker compose helps us in managing containers in docker. You can edit the configuration in docker compose file. Let’s create docker compose files for MySQL and HAProxy containers by following the steps below.
version: '3.9'
services:
mysql-master1:
image: mysql:8.0
container_name: mysql-master1
restart: unless-stopped
env_file: master1/.env
command: --default-authentication-plugin=mysql_native_password
cap_add:
- all
volumes:
- ./master1/my.cnf:/etc/mysql/my.cnf
- ./master1/data:/var/lib/mysql
environment:
- TZ:${TZ}
- MYSQL_USER:${MYSQL_USER}
- MYSQL_PASSWORD:${MYSQL_PASSWORD}
- MYSQL_ROOT_PASSWORD:${MYSQL_PASSWORD}
networks:
default:
aliases:
- mysql
mysql-master2:
image: mysql:8.0
container_name: mysql-master2
restart: unless-stopped
command: --default-authentication-plugin=mysql_native_password
env_file: master2/.env
cap_add:
- all
volumes:
- ./master2/my.cnf:/etc/mysql/my.cnf
- ./master2/data:/var/lib/mysql
environment:
- TZ:${TZ}
- MYSQL_USER:${MYSQL_USER}
- MYSQL_PASSWORD:${MYSQL_PASSWORD}
- MYSQL_ROOT_PASSWORD:${MYSQL_PASSWORD}
networks:
default:
aliases:
- mysql
haproxy:
image: haproxytech/haproxy-ubuntu:latest
container_name: haproxy
restart: unless-stopped
ports:
- "2999:2999"
cap_add:
- all
volumes:
- ./haproxy.cfg:/etc/haproxy/haproxy.cfg
networks:
default:
aliases:
- mysql
Creating MySQL Master 1 Config for Load Balance with MySQL
I have used the latest version of MySQL to create a Master-Master MySQL-Replication on server 1. To make it easier to install MySQL, I used docker and created a persistent volume to allow the data in the container to be stored on the host server.
Additionally, create a persistent volume for my.cnf to connect to the container. Create a master1 folder to store data on MySQL replication on Master 1.
ubuntu@haproxy: mkdir /data/haproxy-mysql/master1

Create a data folder in the Master 1 directory; this data folder is useful for storing persistent volumes from the MySQL master 1 container to the host server.
ubuntu@haproxy: mkdir /data/haproxy-mysql/master1/data

Moreover, we need a custom configuration file in my.cnf on the host server that will create a persistent volume on the container.
ubuntu@haproxy: touch /data/haproxy-mysql/master1/my.cnf

After that, open the my.cnf file and add configuration to my.cnf file to do replication on MySQL Master 1.
# Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; version 2 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
#
# The MySQL Server configuration file.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html
[mysqld]
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
datadir = /var/lib/mysql
secure-file-priv= NULL
server-id=1
binlog_format=ROW
log-bin
skip-name-resolve
# Custom config should go here
!includedir /etc/mysql/conf.d/
Then, create an environment variable (env) file to store the environment username and password in MySQL. As such, this lasts for the duration of terminal.
ubuntu@haproxy:~$ touch /data/haproxy-mysql/master2/.env

Moreover, fill the env file on Master 1 like this:
### WORKSPACE #############################################
TZ=UTC
#MYSQL_DATABASE=master
MYSQL_USER=master
MYSQL_PASSWORD=Master123@
MYSQL_PORT=3306
MYSQL_ROOT_PASSWORD=Mastermaster123
The following is a tree view in the Master 1 directory for MySQL master1 replication.

Create Master 2 MySQL Config File
Since we have made the configuration on mysql master 1, let’s make the configuration on MySQl Master 2 now.
To do this, create a Master 1 folder with this command:
ubuntu@haproxy:~$ mkdir /data/haproxy-mysql/master2

In addition, create a data folder in the Master 2 directory to create a persistent volume on mysql master2.
ubuntu@haproxy:~$ mkdir /data/haproxy-mysql/master2/data

Therefore, don’t forget to create my.cnf file for MySQL Master 2 replication configuration.
ubuntu@haproxy:~$ touch /data/haproxy-mysql/master2/my.cnf

Add the contents of the my.cnf configuration as below.
# Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; version 2 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
#
# The MySQL Server configuration file.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html
[mysqld]
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
datadir = /var/lib/mysql
secure-file-priv= NULL
server-id=2
binlog_format=ROW
log-bin
skip-name-resolve
# Custom config should go here
!includedir /etc/mysql/conf.d/
In addition, create env file for MySQL Master 2.
ubuntu@haproxy:~$ touch /data/haproxy-mysql/master2/.env

Let’s fill in the environment username and password in the env file master2.
### WORKSPACE #############################################
TZ=UTC
#MYSQL_DATABASE=master
MYSQL_USER=master
MYSQL_PASSWORD=Master123@
MYSQL_PORT=3306
MYSQL_ROOT_PASSWORD=Mastermaster123
Congratulations! You have created the replication configuration on MySQL Master 2. You can verify by using the tree:

Make Build Up Container for Load Balance on MySQL
Before you make docker-compose, change the permissions of the Master 1 and Master 2 folders according to the user on your server first with chown.
ubuntu@haproxy:/data/haproxy-mysql$ sudo chown -R $(whoami): master*

Not only that, but also change the data access rights so that it can be read by the container with this command (like the problems in Stack Overflow).
ubuntu@haproxy:/data/haproxy-mysql$ chmod -R a+rwx master1/data/
ubuntu@haproxy:/data/haproxy-mysql$ chmod -R a+rwx master2/data/

We have made all the configurations of docker compose and the mysql folder structure, let’s build using docker compose. We go to the haproxy-mysql directory then run the command “docker-compose up -d”.
ubuntu@haproxy:/data/haproxy-mysql$ docker-compose up -d

Let’s see the service container can run properly with the command “docker-compose ps”.
ubuntu@haproxy:/data/haproxy-mysql$ docker-compose ps

The container is running fine, then we need to make a replication configuration on mysql. We will make a master master replication first before heading to loadbalance with haproxy.
Create Replications on MySQL
This is where you will enter the configuration stage in MySQL. Enter the MySQL Master 1 container to create the haproxy_user user. Then, enter the MySQL container with the root user and password that you added to the master1/.env file.
ubuntu@haproxy:/data/haproxy-mysql$ docker-compose exec mysql-master1 bash

In this step you will create the user haproxy_user on mysql master1 by logging in using the root user firstL
mysql -u root -p

Hence, create a new user “replication” on Master 1 MySQL
CREATE USER 'replication'@'%' IDENTIFIED WITH mysql_native_password BY 'Slaverepl123';

Let’s allow user replication so that you can use it as replication in MySQL.
mysql> GRANT REPLICATION SLAVE ON *.* to 'replication'@'%';

After creating slave on MySQL master1, check master slave status on MySQL Master 1.
mysql> SHOW MASTER STATUS;

Then, go to MySQL Master 2 to create user replications and allow master master replications. Similarly, enter into the mysql-master2 container now.
ubuntu@haproxy:/data/haproxy-mysql$ docker-compose exec mysql-master2 bash
Subsequently, login to the root user on MySQL Master 2.
root@a46ab42e4e7c:/# mysql -u root -p

Create user replications on master2 mysql.
mysql> CREATE USER 'replication'@'%' IDENTIFIED WITH mysql_native_password BY 'Slaverepl123';

After that, grant replication slave on Master2 MySQL with user MySQL.
mysql> GRANT REPLICATION SLAVE ON *.* to 'replication'@'%';

Congratulations! You have created user replications and granted replication slaves on MySQL Master 2. To confirm, let’s see the status of the master.
mysql> SHOW MASTER STATUS;

Now, stop the slave on MySQL Master 1.
mysql> STOP SLAVE;

Accessing The Slave Logs on Master 1
You can access the initial bin log and slave position on MySQ with master slave log on MySQL Master 1.
mysql> CHANGE MASTER TO MASTER_HOST = 'mysql-master1', MASTER_USER = 'replication', MASTER_PASSWORD = 'Slaverepl123', MASTER_LOG_FILE = '6e360f05f522-bin.000001', MASTER_LOG_POS = 673;

Then, start the slave on MySQL Master 1.
mysql> START SLAVE;
After doing that, check the status of the slave on MySQL Master 1.
mysql> SHOW SLAVE STATUS \G;

Accessing The Slave Logs on Master 2
You can access the initial bin log and slave positions on MySQL 1 with master slave log on MySQL Master 2.
mysql> CHANGE MASTER TO MASTER_HOST = 'mysql-master2', MASTER_USER = 'replication', MASTER_PASSWORD = 'Slaverepl123', MASTER_LOG_FILE = 'f9ce66dff73f-bin.000001', MASTER_LOG_POS = 673;

In addition, start slave on MySQL Master 1.
mysql> START SLAVE;
You can examine the slave status on MySQL Master 1 now.
mysql> SHOW SLAVE STATUS \G;

Likewise, now you can stop the slave on MySQL Master 2.

Creating The HAProxy User for HAProxy docker container to use.
With this step you will get to create the user haproxy_user on Master 1.
mysql> CREATE USER 'haproxy_user'@'%';

Likewise, create a user haproxy_root– where only haproxy instances will be able to use the haproxy_root user.
mysql> CREATE USER 'haproxy_root'@'%' IDENTIFIED WITH mysql_native_password BY 'Haproxymysql123';

To confirm this step, write a query to fetch the users list in MySQL Master 1 which will be synchronized to MySQL Master 2.
mysql> SELECT DISTINCT User FROM mysql.user;
Here’s how the output looks on MySQL Master 1.

Similarly, this is the output on MySQL Master 2.

HAProxy MySQL; Configure The HAProxy to Load Balance
How cool is that? You have successfully created a MySQL replication with Master-Master for your haproxy mysql setup.
Therefore, the logical next step is to configure haproxy to make MySQL Loadbalance with TCO.
To do this, you will have to enter the haproxy container.
ubuntu@haproxy:/data/haproxy-mysql$ docker-compose exec haproxy bash

Great! With that done, let’s start configuring loadbalance on haproxy.
Firstly, open haproxy configuration path at /etc/haproxy/haproxy.cfg.
root@016fbb5028ee:/# vi /etc/haproxy/haproxy.cfg

Then, add the configuration to the path /etc/haproxy/haproxy.cfg shown below. This will be used by the HAProxy docker container.
#---------------------------------------------------------------------
# Example configuration for a possible web application. See the
# full configuration options online.
#
# https://www.haproxy.org/download/1.8/doc/configuration.txt
#
#---------------------------------------------------------------------
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
# to have these messages end up in /var/log/haproxy.log you will
# need to:
#
# 1) configure syslog to accept network log events. This is done
# by adding the '-r' option to the SYSLOGD_OPTIONS in
# /etc/sysconfig/syslog
#
# 2) configure local2 events to go to the /var/log/haproxy.log
# file. A line like the following can be added to
# /etc/sysconfig/syslog
#
# local2.* /var/log/haproxy.log
#
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
# turn on stats unix socket
stats socket /var/lib/haproxy/stats
# utilize system-wide crypto-policies
ssl-default-bind-ciphers PROFILE=SYSTEM
ssl-default-server-ciphers PROFILE=SYSTEM
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
#mode http
log global
option tcplog
retries 2
timeout connect 4s
timeout client 30m #1440m
timeout server 30m #1440m
timeout check 10s
#maxconn 3000
listen stats
mode http
bind *:2999
stats enable
stats uri /stats
stats realm Strictly\ Private
stats auth devops:Haproxymysql2021@
listen mysql-cluster
mode tcp
option tcpka
bind *:3306
option mysql-check user haproxy_user
balance roundrobin
server master1 mysql-master1:3306 check
server master2 mysql-master2:3306 check
Here, it is crucial to validate the haproxy configuration with this command below:
root@016fbb5028ee:/# haproxy -c -V -f /etc/haproxy/haproxy.cfg

After confirming that the haproxy configuration is valid, restart the haproxy container so that the TCP Loadbalance on port 3306 can run on haproxy.
ubuntu@haproxy:/data/haproxy-mysql$ docker-compose restart haproxy

After restarting, check if port 3306 is running on the haproxy container.
root@016fbb5028ee:/# netstat -ntlp

In the same way, try to test with the MySQL client– to test it we need to install MySQL client on haproxy.
root@016fbb5028ee:/# apt update && apt install mysql-client -y

In the end, check for the server_id using this command:
root@8e949cb0bda6:/# mysql -u root -p -h 127.0.0.1 -e "show variables like 'server_id'";

Note: You can also check HAProxy docker container stats on the browser using Server IP.

Conclusion
In conclusion, it is safe to say HAProxy is quite useful for HTTP and TCP Loadbalancers. This way we can use it for TCP loadbalance on MySQL with Master-Master replication on MySQL. Besides being useful for simplifying the development process, you can also make MySQL work powerfully. I hope you liked this post and were able to follow along.
Finally feel free to check out the HAProxy Load balance using MySQL code on github.
Edited by: Syed Umar Bukhari