Using a physical machine as if it were an on-demand Virtual Machine

Here are the specs for the best machine I have in the house:

• Intel 13th Gen i3-13100
• 32GB of DDR4 RAM
• Nvidia Tesla P40 (24GB of VRAM)

This machine lives in a 4U rack case in my server rack along with the rest of my homelab.

While my 2019 Macbook Pro technically has better specs, as soon as it starts to do any work it attempts to melt the surface it is sitting on (this was the last Intel based model made).

So I figured, using VS Code’s awesome remote development extension, I could use my Macbook as a terminal and push the hard work onto the better machine by SSHing in, especially when I want to do any work involving local LLMs.

To avoid running the machine in the rack 24/7 (especially since I pay the electricity bill), I wondered whether it would be possible to turn the machine on and off only when I wanted to use it. I had already configured it to use Wake On Lan which meant I didn’t need to phyiscally push the power button, but it still meant that I had to trigger the WOL packet from within the local network (which I eventually discovered was an issue when I was using my VPN from a remote location).

I also had left it running without shutting it down a few times, which was frustrating to discover after the fact. I wished it would behave more like an on-demand / ephemeral Virtual Machine, where I only paid for it when it was being used.

To solve this, I wrote ssh-wol-proxy. This is a tiny server written in Go that I run within my homelab. It listens for incoming SSH connections and when a connection is established, it sends a WOL packet to my machine. It then makes multiple attempts to establish a TCP connection on the target SSH port. So as long as your SSH connection timeout is longer than it takes for the machine to start and for sshd to become available, then the proxy will manage to connect and connect you to the machine.

While this was a great solution, it did not solve the part where I left the machine running. For that I configured debian-autoshutdown to check every five minutes whether there is any network traffic on port 22. If after a few checks there is no traffic, it shuts down the machine.

These two solutions together solve my issues. I now have a physical machine that behaves in an on-demand way. I only need to open VS Code and connect to the proxy using the remote development extension and within approximately 45 seconds I’m ready to work.

The code could be greatly improved, it suffers from multiple race conditions, but in a homelab setting it works just fine.