First: what are we talking about?
Note : Ceci est la version anglaise de l'article, pour la version française, voir ici.
Haproxy is a proxy software. It has many use, but here we will use its capacity to reverse proxying HTTP and HTTPS.
For this post, we will consider you have a working Haproxy server and a working configuration.
Let's Encrypt :
Let's Encrypt is an open source project sponsored by Mozilla foundation and Cisco.
Idea is to create an free certificate authority in order to create SSL certificate and doing so, allowing to most people possible to use HTTPS and secure websites.
Regarding usual cost of SSL certificate, this a very interesting project!
In addition to that, Let's Encrypt is really easy to use, mainly thank to an API allowing to create certificate very easily.
We will use this API in this post through Let's Encrypt official python client.
EDIT : 04/04/16 : add note about case when you use redirect HTTP to HTTPS in your HTTP haproxy frontend
EDIT : 09/16/16 : Update with new binary + add git repository with some scritps to automate LE management
Let's Encrypt offers many option to create and validate certificate via its client.
If your Haproxy is localised on the same server than your web server, you can use the --webroot option, which allow Let's Encrypt to store a special file directly in the root directory of your website, in order to allow Let's Encrypt server to request the file and validate that you are the real owner of the domain.
If you want to use this option, you can read this blog post about using webroot and haproxy, including using a dedicated apache webserver for validation.
For this post, we will use instead the --standalone option, which launch a mini-webserver. This webserver will be used for the validation process, as Let's Encrypt server will request it directly. Usually, this option has a big disadvantage: as the webserver bind on 80 network port, the real webserver needs to be temporally shutdown during the validation process.
In other word, your website(s) will be unavailable during all the process.
But we are going to use an option to change the default port, in coordination with haproxy frontend/backend capabilities, to allow validation without any downtime :-)
-- Many thanks to coolaj86 as his post give me this idea to use haproxy and Let's Encrypt together ;-) --
Get Let's Encrypt client
As LE is now officially released, a new official binary is available, more advanced and more easy to use than the previous letsencrypt-auto
All information are available on the certbot official website, but here are quick instructions how to use it.
chmod a+x certbot-auto
You can next directly use the binary to get your new certificate (at least, once you have configured as this blog article describes it :-p)
./certbot-auto certonly --domains blog.victor-hery.com --renew-by-default --http-01-port 63443 --agree-tos
To avoid any downtime, we will use your existing frontend.
Let's Encrypt will request the IP address which resolve your website, so if you have many frontends or many IPs, you need to configure each frontend according to your needs.
Main idea is : during the process, Let's Encrypt request the base website URL, following by /.well-known/acme-challenge/a_unique_id.
So we are going to configure a haproxy ACL which match this path to redirect it to a specific backend!
frontend http-in acl app_letsencrypt path_beg /.well-known/acme-challenge/ [...] use_backend bk-letsencrypt if app_letsencrypt
- path_beg: match the path (the part just after the first /) that begin by .well-known/acme-challenge/
Doing so, all Let's Encrypt requests will be redirected to the bk-letsencrypt backend.
Warning : if you are using redirect from HTTP to HTTPS for your website, haproxy will also redirect Let's Encrypt request to your HTTPS frontend.
You will then need to add the acl and use_backend lines to your HTTPS frontend as well, or let's encrypt will get 404 not found answer.
About the backend, we are going to configure it to redirect request to the server launched by Let's Encrypt client.
backend bk-letsencrypt log global mode http server srv_letsencrypt 127.0.0.1:63443
- mode http: allow to check the HTTP consistency of the request
- server: this line redirect to the server that Let's Encrypt client will launch on localhost, port 63443
The local server will not always be up, only when the client is running. Rest of the time, haproxy will return a 503 error if someone try to get an URL matching the ACL.
Of course, you need to reload haproxy after doing these modifications.
systemctl reload haproxy.service``` ## Configure and use of Let's Encrypt #### Configuration : <p>To simplify the command line usage, we use a configuration file for Let's Encrypt client.</p> <p>By default, the client uses the file <em>/etc/letsencrypt/cli.ini</em>. So this is the file we are going to edit.</p>
rsa-key-size = 4096 email = your_admin_email authenticator = standalone standalone-supported-challenges = http-01
<ul> <li>rsa-key-size: tell letsencrypt to direclty generate 4096 bits certificate, more strong that default 2048. You can downgrade to 2048 (but never less!) if your server is low performance to gain some generation time</li> <li>email: use a valid email address, as it will be used if certificate recovery is needed through the Let's Encrypt website.</li> <li>authenticator: as seen before, we will use <em>standalone</em> mode</li> <li>standalone-supported-challenges: this option is specific to standalone mode, and allow to choose the method used for the verification process, between <em>http-01</em> and <em>tls-sni-01</em>. Here we use http-01 as our website has no valid certificate (the first time), and so haproxy will not have valid SSL certificate to use in its frontend for Let's Encrypt server request.</li> </ul> <p>Using HTTP for the verification process is safe, as only the verification request will be send in clear, and it has no secret inside.</p> ### Generate the certificate <p>The command line to use to generate your certificate is the following:</p>
/root/letsencrypt/certbot-auto certonly --domains yourdomain.tld --renew-by-default --http-01-port 63443 --agree-tos```
- certonly: tell the client that we only want to generate the certificate, not using some installation plugin to install certificate somewhere.
- domains: the domain or subdomain for which you want your certificate
- renew-by-default: tell letsencrypt that, if the certificate already exist, it should renew it. If it does not exist, it will create it.
- http-01-port: tell the network port to use for the temporary validation server launched. This is the port used in the haproxy backend we have configured before.
- agree-tos: tell the client to automatically accept the therm of service of Let's Encrpt (If you are here, you have read it and agree, right ?)
Installing the certificate
Certificates are created in a directory called /etc/letsencrypt/live/yourdomain.tld/ by the client during the generation process.
You will find multiple files inside:
- cert.pem : the certificate public part (crt)
- chain.pem : the authority chain (ca of authorities)
- fullchain.pem : a concatenation of cert.pem and chain.pem
- privkey.pem : the certificat private key
In order to use with haproxy, you need to concatenate fullchain.pem and privkey.pem, and store it where haproxy read its certificates.
cat fullchain.pem privkey.pem > domain.tld.pem``` <p>For example for the following HTTPS frontend:</p>
frontend https-in bind IP:443 ssl crt /etc/haproxy/cert/
<p>Here we need to store the domain.tld.pem file in <em>/etc/haproxy/cert/</em></p> <p>Once your certificate is stored in the right place, reload haproxy for it to re-read certificate, and everything should be OK.</p> <p>Of course do not forget to configure an HTTPS frontend with correct ACL for your website in haproxy!</p> <p>Usually, simply copy/paste of your HTTP frontend changing port to 443 (and adding ssl and crt option) will do the job.</p> ## Limitations ### Renew the certificate <p>Even from the end of beta time, LE has taken decision to provide certificate for 90 days duration max. It allow them to avoid abuse usage and to make the project alive by forcing regular renew.</p> <p>Note that unlilke the beta client, certbot-auto is capable to use all your available certificates (thanks to the directory<em>/etc/letsencrypt/renewal/</em>) to allow automatic renewal .</p> <p>So remember to configure a cron job on your server to renew certificate:</p>
30 01 01,10,20,30 * * /root/letsencrypt/certbot-auto renew```
This cron task will launch the renew command each 10 days to be sure your certificates will stay valids.
Do not forget that you still need to create the .pem from fullchain.pem and .key to give it to haproxy. See below for some automation about this task!
Rate limit of certificate by domain
Because of its free use Let's Encrypt use a rate-limit sysytem to avoid generation of too many certificate for the same domain.
You will get an error if you try to generate too quickly or too often certificates for the same domain
Be careful: the limitation take into account subdomain as well! All certificates finishing with domain.tld will be count.
For the moment, Let's Encrypt does not allow certificate for international Domain Name.
This is impossible to generate a certificate for a domain or subdomain containing accent or special characters.
For example, impossible to generate certificate for https://blog.héry.com unfortunately.
There is currently a boulder on Let's Encrypt github, but it is still no available. If you are willing to help or need the function, does not hesitate to join :-)
Bonus: some scripts
In order to make the process easier, I have written some scripts to allo generation and renewal of certificate, and haproxy interactions, easier.
They are available in my git repository, so you can easily clone them in your server:
git clone https://git.lecygnenoir.info/LecygneNoir/letsencrypt-haproxy.git
README simply describes how to use them, it pretty simple.
create-certificate allow you to create a certificate for the domain you pass to the script, then it creates the .pem for haproxy, store it in the given directory and reload haproxy.
renew-certificates only renew all certificates that need to be renewed, creates as well haproxy pem files, en reload haproxy. You can use renew-certificate in our cron task as explained before if you want.
Do not forget to check path in scripts, mainly where to store certificates for haproxy, and path to certbot binary
And voila, with all of that, you should be able to create all certificates you need, and use them directly in haproxy, without any downtime! So, welcome in the HTTPS world :-)
Si vous avez des questions, si quelque chose n'est pas clair, n'hésitez pas à commenter !
Hi, thank you for this post. I have implemented this a while ago, but have one issue you might have solved. When I have reissued the certificates these need to be reloaded by haproxy (I run 1.5) but a
... haproxy -f ... -p pidfile -sf pidfile
does not seem to (always) reload the certificates.
I never hit this problem for the moment.
It's possible you hit the Haproxy feature that do not stop directly the haproxy process, but instead spam a new one and let the older(s) continue to deals with opened connections.
If you use Haproxy with TCP connections (such as handling pop, smtp or imap, ftp, etc) haproxy could potentially never ends its older processes. Perhaps it is the cause of your problem ?
I advice to check if your haproxy process has really restarted with ps faux | grep haproxy
And try to restart haproxy instead of simply reloading it.