Disable HTTPS for one hostname on server with other HTTPS hosts

I was nerdsniped by some post on StackExchange that claimed it is impossible to disable HTTPS for a single hostname if you have other HTTPS hosts on that server.

But this is possible! If you have a modern version of nginx, you can use the following:

server {
    listen *:443 ssl default_server http2;
    listen [::]:443 ssl default_server http2;
    server_name _;

    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_reject_handshake on;
}

But if you are using an older (before 1.19.4) version of nginx that does not have the ssl_reject_handshake directive or you’re using a different webserver that does not have any of this, you can use a different approach. You need to get the hostname from the SNI field in the TLS ClientHello packet and then do a TCP reset. In Firefox, you’ll get the ‘Secure Connection Failed’ error page, instead of ‘Warning: Potential Security Risk Ahead’. I think the first one is a more user friendly error.

You can use the xt_tls project: https://github.com/Lochnair/xt_tls

Use the following iptables rule to disable access to a specific host. Replace example.org with the hostname you are serving.

sudo iptables -A INPUT -p tcp --dport 443 -m tls --tls-host "example.org" -j REJECT --reject-with tcp-reset

The nice thing is that it works for multiple webservers, but the not so nice thing is that it happens outside of your webserver configuration. Be careful with putting some random GitHub project into your kernel. If feasable, try to upgrade or switch to nginx :)