Stage WS: locking the front door before building it

Stage 5 gives Waystone a web panel: a login page, and buttons for start, stop and the operation history. That means the controller will answer browsers for the first time. Before writing a single page, I wanted the door they come through to be solid. Stage WS is that door. There's no UI in it yet. It's the part every page will stand on.

Like stage 4c, it came in two halves. WS-1 is the code and tests. WS-2 is a real installation on the dev VM, tested from the VM and from a browser on my PC.

The shape

Browsers never talk to the controller directly. They talk HTTPS to Caddy, and Caddy forwards each request to a private Unix socket, web.sock. The controller has no TCP port at all, in any build. It now has three sockets, each with its own job:

• api.sock for local admin tools (stage 4),
• agent.sock for the agent (stage 4),
• web.sock for browsers, through Caddy.

systemd owns web.sock: root:waystone-web, mode 0660, and only Caddy's user is in that group. On top of that, the controller asks the kernel who is on the other end of every connection, and accepts exactly one user id: Caddy's. Root and the controller's own user are refused too. On the VM, a request to web.sock from my own account, from the agent's account and from root are all refused before a single byte of HTTP is read.

One front door, one guard

Every browser request passes the same guard, and there's only one way to build the browser side of the controller, through that guard. A test scans the code for any route that tries to get around it.

The guard checks the request against one setting, WAYSTONE_PUBLIC_URL, the address the panel is meant to have. It never trusts what the request says about itself:

• Wrong host name: 421. Waystone only answers for its own name, so a request aimed at some other name gets nothing.
• Forwarding headers: Caddy sets exactly three, saying the request came in over HTTPS, for this host, from this client address. Each must appear exactly once, with a value that makes sense, or the request gets a 400.
• Headers that shouldn't be there: Forwarded, X-Real-IP, CF-Connecting-IP and a handful of others. Caddy strips them, so if one shows up, the proxy isn't set up the supported way, and the request gets a 400 instead of a guess.
• Changing something (anything but a plain read) must come from the panel's own page. A request from another site, or one that won't say where it came from, gets a 403. Stage 5 adds proper CSRF tokens on top. This check doesn't replace them.

Every response carries a strict Content-Security-Policy, refuses to be framed, and isn't cached. HTTPS responses carry HSTS, starting at five minutes, so a setup mistake is cheap to undo. You raise it once everything works.

If the public URL is missing or isn't plain https://, the controller refuses to start and says exactly which setting is wrong. It never falls back to something looser.

How WS-1 was tested

Every rule above has table-driven tests, including tests that run over a real Unix socket and check the kernel-level user id refusal for real.

Then I broke the code on purpose, 161 different ways. 37 survived the first round, and each real gap got a test. Of the 9 left at the end, 7 can't change behaviour, because another check already catches the same thing. The last 2 were "the web server isn't wired into the controller at all", which only a real install can catch. WS-2 did.

Everything passes three times in a row under Go's race detector, locally and on the dev VM, and builds for Windows. Release binaries contain no trace of the development mode.

WS-2: the real thing

On the dev VM: Caddy, with a certificate from a small development certificate authority I made on my PC. That CA is limited by its own certificate to names under waystone.test, so even if its key leaked, it couldn't vouch for any other site. Its key never leaves my PC.

The gate script tests from the outside in, against the real Caddy, socket and controller:

• https://panel.waystone.test/healthz answers with every security header. Plain http redirects to https, and the bare IP address gets no Waystone content at all.
• A request through Caddy stuffed with forged forwarding headers still works, and Waystone's log shows the real client address, not the forged one. A tiny recorder behind a second Caddy shows exactly what Caddy sends upstream: three headers, nothing else.
• Talking to web.sock as Caddy's user, 22 hand-made bad requests each get the right answer: 421, 400 or 403.
• A broken public URL makes the controller stop cleanly with a clear message. Putting it back brings everything back.
• All of stage 4 still works: start, restart, stop, queuing while the agent is down, and server_busy during a live stop.
• After a reboot, with nothing touched, everything comes back and passes again.

The final runs were 82 checks and then 28 after the reboot, all passing. Then I opened the panel's health page in Firefox on my PC: no warnings, padlock from the dev CA, and every header in the network tab.

What the install found

The first gate run had 10 failures. None of them was the guard, but they were worth the evening:

• systemd won't start a new socket for a service that's already running. On an upgrade, stop the controller first, then start the socket, then the controller. The deploy guide says so now.
• My test was wrong. To test a bad header, it added a second copy on top of the good one, and curl only sends the first. So those tests never sent the bad value. Each case now changes exactly one thing. With that fixed, every one passed.
• Caddy 2.6.2 with a bind line only put its automatic http→https redirect on one address. An explicit redirect block fixes it.
• Ubuntu's Caddy is old enough not to have trusted_proxies. So the misconfiguration test now covers the mistake this Caddy can actually make: leaving out the forwarding lines. Caddy still replaced the forged address.

What the screenshots show

This is the dev VM after the WS-2 install. My username is covered in the prompts and in the two "directly" lines, and the hostname is covered in the prompts. The faint page behind the terminal is this website. From top to bottom:

1. Three sockets. agent.sock, api.sock and web.sock all belong to root, with the groups waystone-agent, waystone-admin and waystone-web, and nobody else has any access.
2. Everything running. Caddy, the web socket, the controller and the agent are all active.
3. The controller's log. It's listening on web.sock for https://panel.waystone.test, and it will accept only the proxy's user id, 980.
4. The health page over HTTPS. HTTP/2 200 with every header from "One front door, one guard": no caching, the Content-Security-Policy, no framing (frame-ancestors 'none' and X-Frame-Options: DENY), nosniff, same-origin policies, and HSTS at max-age=300, the five-minute starting value. The body is {"status":"ok"}.
5. Plain http. A 301 to the same address over https.
6. Hand-made requests to web.sock as Caddy's user. Correct headers get 200. The wrong host name, evil.test, gets 421. Claiming the request came over plain http gets 400, and so does a forwarded-for header with two addresses in it.
7. Two locks, even for root. My own account gets curl exit 7, "couldn't connect": web.sock is root:waystone-web with mode 0660, my account isn't in that group, so the kernel refuses the connection and Waystone never even sees it. Root ignores file permissions, so its connection opens, but then the controller asks the kernel who's on the other end, sees user 0 instead of Caddy's 980, logs the refusal and hangs up. That's exit 56, "the connection broke while receiving". The file permission is the first lock and the user check is the second, and not even root gets past both.
8. The real address, not the forged one. The last request carried a forged X-Forwarded-For: 6.6.6.6 and X-Real-IP: 6.6.6.6, yet the log records client_ip="127.0.0.1". That's the truth: the request ran on the VM and reached Caddy over loopback. Caddy writes the address it actually saw into X-Forwarded-For, throws the forged one away and strips X-Real-IP. From my PC's browser, the same line would show my PC's address on the network.

The second screenshot is the same health page in Firefox on my PC. There are no warnings, and the padlock says the certificate was verified by the Waystone dev CA, limited to waystone.test. Firefox points out that Mozilla doesn't know this issuer. That's expected, because it's my own development CA, which I added to my PC myself.

What's next

Stage 5: the browser login, sessions and CSRF tokens, then the panel itself, all behind this door. The Caddy config and a setup guide are in deploy/.

Waystone is open source under AGPL-3.0.