Use Cloudflare HTTPS with Rancher and HA Proxy

I have my own personal PaaS platform that is powered by Rancher. It consists of 3 cheap VPS from OVH and one database hosted in AWS. One VPS is used to run Rancher, the two others as Docker hosts to be used by Rancher to run my applications.

On top of that, I use Cloudflare as a Name Server for my personal (leward.eu) domain and domains related to the apps running on my platform.

Cloudflare is also enabled as an HTTP proxy for some of my services, mainly to easily enable HTTPS.

The Setup

I have three domains I want to use:

• leward.eu
• app-a.leward.eu
• app-b.leward.eu

I have three VPS servers:

• Rancher
• Docker Host #1
• Docker Host #2

My domains are configured to point to all my docker hosts. The DNS records looks like this (simplified):

;; A Records
leward.eu           A       IP_DOCKER_HOST_1
leward.eu           A       IP_DOCKER_HOST_2
platform.leward.eu  A       IP_RANCHER_HOST

;; CNAME Records
app-a.leward.eu     CNAME   leward.eu.
app-b.leward.eu     CNAME   leward.eu.

On each docker Host I have an HA proxy listening on port 80 and directing the traffic to the proper Docker container based on the HTTP Host Header. This works even if the app is not on the same docker host as the HA proxy instance because Rancher handles the networking between two hosts.

The domains are managed by Cloudflare, with their HTTP proxy enabled and forcing the traffic to HTTPS.

A request made to one of my application actually going throught the following layers:

User --[HTTPS]--> Cloudflare --[HTTP]--> HA proxy --[HTTP]--> App

This not perfect, but it is quite convenient and removes the hassle of having to deal myself with certificates.

I would like in the future to be able to have HTTPS all the way (maybe using Let’s Encrypt):

User --[HTTPS]--> HA Proxy --[HTTPS]--> App

Diagram made with draw.io and their amazing SVG export feature

When relying on Round-robin DNS to achieve load balancing and high-availability, it is good to use Floating IP addresses that you can easily map to another host.

The Application believes it is accessed through HTTP

I have a Grails application running with this configuration. And when I used the Grails redirect method it did not work because the URL built was using HTTP and not HTTPS.

The issue is that HA Proxy itself is listening in HTTP, so when forwarding the traffic in indicates it is running expose and exposed through plain HTTP.

There is a trick that solved the problem: adding an X-Forwarded-Port HTTP header at the HA Proxy level.

Configure HA proxy with Rancher

Custom configuration can be added to HA Proxy on Rancher. However, figuring out how to properly apply the configuration was not straightforward.

Let’s say App B is the application on which I want to correct that behavior.

Then I have to add in my custom haproxy.cfg:

backend 80_app_b_leward_sg_
    http-request add-header X-Forwarded-Port 443

80_app_b_leward_sg_ is a value generated by Rancher. You can verify which one Rancher has generated for your application by running a Shell into the HA Proxy container and inspect the configuration using vi /etc/haproxy/haproxy.cfg

After the configuration being applied, my Grails app could properly perform the redirect using HTTPS.