Chapter 03 · The session plane

Mosh & Blink Shell

SSH was designed for a workstation with a cable in the wall. A phone changes IP address when you leave the house, and iOS freezes applications the moment you switch away from them. This chapter is about the protocol that treats those events as normal.

Why SSH dies on a phone

SSH runs over TCP, and a TCP connection is identified by four values: source IP, source port, destination IP, destination port. Change any one of them and it is a different connection — the kernel has no way to say "this is the same conversation from a new address."

So a phone hands SSH three unavoidable problems:

  • Roaming. Wi-Fi to LTE changes your source IP. The 4-tuple is invalid. Dead.
  • Suspension. iOS aggressively freezes backgrounded apps. A frozen app stops reading its socket; eventually the OS tears it down.
  • Termination. Under memory pressure iOS kills the app outright. The socket dies without a clean shutdown.

That last case has a signature worth recognising, because it tells you the fault is not on the server. A process that closes a socket properly sends a FIN and the server logs a clean disconnect. A process that is killed leaves the kernel to send a RST, and the server logs: 

Read error from remote host 100.126.245.94 port 50313: Connection reset by peer

Meanwhile the client prints the message you have probably seen far too often: 

client_loop: send disconnect: Broken pipe
Read the pair together

If the server log says Received disconnect ... disconnected by user, the session ended cleanly and the cause is elsewhere. If it says Connection reset by peer, the client was killed and no amount of server tuning will fix it. Keepalives make the server patient; they cannot revive a socket the phone has already lost.

What Mosh does differently

Mosh (mobile shell) discards the idea that a terminal session is a byte stream over a connection. Instead, both ends model the terminal as a screen state, and they synchronise that state over UDP datagrams. The protocol is called SSP, the State Synchronisation Protocol.

SSH over TCP connection = the 4-tuple phone 10.0.0.4:51200 sshd :22 — you walk outside · Wi-Fi → LTE — phone 100.82.7.9:44100 sshd unknown 4-tuple → not this connection connection destroyed shell killed · unsaved work gone client_loop: send disconnect: Broken pipe Mosh over UDP session = shared key + screen state phone 10.0.0.4 mosh-server holds the screen — you walk outside · Wi-Fi → LTE — phone 100.82.7.9 mosh-server right key → same session, new address session continues server just notes the new address the address changed · the session did not
  1. Both are workingYou are on home Wi-Fi with a shell open. Nothing here distinguishes the two — this is the state you spend most of your day in, and it is the only state where SSH and Mosh look the same.
  2. 1 · You leave the houseThe phone drops Wi-Fi and joins LTE, so its source address changes from 10.0.0.4 to 100.82.7.9. Nothing has broken yet. Both sides still believe they have a session.
  3. 2 · The next packet arrivesThis is the moment the two protocols diverge. To the server's TCP stack the new 4-tuple is simply not the connection it is holding — it has no way to express "same conversation, new address," so the packet belongs to nothing. Mosh asks a different question: does this datagram carry the right AES-OCB session key? It does, so the packet is genuine, and the server updates its idea of where you are.
  4. 3 · What you seeSSH gives you Broken pipe and a dead shell. Mosh gives you nothing at all — no reconnect, no message, no pause, because from the protocol's point of view nothing notable happened. The address is not part of the session's identity, so changing it is not an event.
Authentication is per-datagram, not per-connection. Every Mosh packet is encrypted and authenticated with AES-OCB under a session key that SSH delivered at startup. The server accepts a packet because it carries the right key, not because it came from an expected address — so a new IP is a non-event. There is no connection to break.

The two consequences you feel immediately

It skips instead of retransmitting. If you run cat on a huge file and packets drop, TCP dutifully retransmits every missed byte and your terminal crawls through output you no longer care about. Mosh only ever synchronises to the latest screen state, so it jumps straight to the end. Ctrl-C is instant even mid-flood, because the interrupt does not queue behind a backlog.

It predicts your typing. Mosh echoes keystrokes locally when it is confident about the result, underlining anything it has not yet had confirmed by the server. On a 300 ms satellite link this is the difference between usable and infuriating. When a prediction turns out wrong, it corrects on the next update.

Situation
SSH · a TCP byte stream every byte, in order, or nothing moves server cat huge.log bash 20% loss $ cat huge.log 09:41:02 GET /api/v1/items 200 09:41:02 GET /api/v1/items 200 09:41:03 POST /api/v1/orders 201 09:41:03 GET /api/v1/items 200 streaming · 240,000 lines to go 09:41:03 POST /api/v1/orders 201 [ retransmitting segment 812 ] nothing new can be drawn until the missing bytes arrive, in order the screen is ~40 s behind the file the backlog only grows ^C ( nothing happens ) the interrupt is queued behind 40 s of output you no longer want Ctrl-C is not instant $ ▌ idle · every key will cost a round trip keystroke → server $ ▌ you typed: git status still blank · the keys are in flight ← echo, one full round trip later $ git status▌ it appeared 300 ms after you typed it $ git status▌ every character pays the same 300 ms typing feels like wading Mosh · a synchronised screen only the newest state has to arrive mosh-server cat huge.log bash 20% loss $ cat huge.log 09:41:02 GET /api/v1/items 200 09:41:02 GET /api/v1/items 200 09:41:03 POST /api/v1/orders 201 09:41:03 GET /api/v1/items 200 streaming · 240,000 lines to go 09:47:55 GET /api/v1/items 200 [ jumped to the newest screen ] the missed frames were never resent because no one wanted to read them you are at the tail of the file loss costs detail, never time ^C $ ▌ the interrupt travelled in its own datagram, not behind the backlog Ctrl-C is instant $ ▌ idle · the same 300 ms link keystroke → server $ git status▌ echoed locally, at once underlined = predicted, not yet confirmed by the server 0 ms ← confirmation, 300 ms later $ git status▌ the server agreed · the underline quietly goes away you never waited $ git status▌ a wrong guess is repainted on the next update — Mosh only predicts when it is confident typing feels local
  1. Output is pouring outYou ran cat on a 240,000-line log by accident. Both sessions are drawing it as fast as the link allows, and so far they look identical.
  2. 1 · The link starts droppingYou step into a lift, or the satellite hop degrades. One packet in five never arrives. Both protocols now have to decide what a lost packet means.
  3. 2 · Retransmit, or skipTCP's contract is that every byte is delivered in order, so SSH cannot draw line 813 until line 812 has been re-sent. The backlog grows faster than the link drains it and your screen falls further behind real time. Mosh's contract is different: it only promises the current screen. Frames that were superseded before they arrived are simply never sent again, so it jumps to the tail.
  4. 3 · You press Ctrl-CUnder SSH the interrupt is a byte in the same stream, so it waits behind everything already queued — which is why Ctrl-C feels broken mid-flood. Under Mosh it goes out immediately and the next screen state comes back showing a prompt. This is the difference you feel most often in practice.
  5. A slow link, and a promptSame two sessions, nothing running, 300 ms of round-trip time between you and the server — a satellite link, a bad train connection, or just a long way away.
  6. 1 · You type git statusSSH sends the keystrokes and shows you nothing, because in a terminal the server decides what your characters look like on screen — echo is a server-side decision. Mosh keeps its own model of the terminal, so when it is confident what a character will do it draws it straight away and underlines it to mean "not confirmed yet".
  7. 2 · The round trip completesSSH's echo arrives and the text appears, 300 ms after your finger moved. Mosh's confirmation arrives too, and all it does is remove the underline — you had already been reading the text for a third of a second.
  8. 3 · When the guess is wrongMosh predicts only where it is confident, and a wrong prediction is overwritten by the authoritative screen on the very next update. The worst case is a character that flickers; the ordinary case is a shell that feels like it is running on your phone.
Both consequences come from the same design choice. Because Mosh synchronises a screen rather than a stream, a lost packet costs you a frame you no longer need instead of a delay, and because the client keeps its own model of that screen it can render ahead of the server and correct itself later. Neither trick is available to anything built on a byte stream.
What Mosh does not do
  • No port forwarding. Keep plain SSH around for tunnels — or publish the port to your tailnet with tailscale serve. Chapter 05 works through both, because "my dev server is on loopback and I am holding a phone" is a problem this guide otherwise creates and does not solve.
  • No native scrollback. It draws a screen, so scrolling is tmux's job, not the terminal's.
  • It needs UDP. On the tailnet that is fine; WireGuard is UDP already.
  • It does not survive a server reboot. Only tmux does that.

How a Mosh session starts

Mosh is not a replacement for SSH — it uses SSH as its bootstrap, which is why all your SSH hardening still applies:

Client (Blink) Server 1 · ssh macbook · TCP 22 · publickey auth mosh-server starts picks UDP 60001, detaches 2 · returns the port + a one-time AES-OCB key 3 · the SSH connection closes — it has done its job 4 · SSP over UDP · every datagram authenticated with that key carried inside WireGuard · no public UDP port is opened
  1. You type mosh macbookNothing is listening on UDP yet and no secret exists. Everything that follows is set up by SSH, which is the point — Mosh adds no new way to get in.
  2. 1 · An ordinary SSH loginThe mosh client runs ssh for you and authenticates exactly as it always does: port 22, your public key, your known_hosts check. Every bit of SSH hardening you did in the last chapter applies here unchanged.
  3. 2 · mosh-server starts and reports backOver that SSH connection the client launches mosh-server. It picks a free UDP port between 60000 and 61000, generates a one-time AES-OCB key, prints both to stdout, and detaches into the background. The key travels back down the already-encrypted SSH channel, so it is never exposed.
  4. 3 · SSH closesIts job is finished. There is no long-lived SSH connection propping the session up, which is exactly why nothing breaks when your IP changes later — the fragile TCP connection is already gone by the time you leave the house.
  5. 4 · The session runs over UDPThe client starts speaking SSP to that port, authenticating every datagram with the shared key. From here on there is no TCP connection anywhere in the picture, and no server-side state that depends on your address.
SSH is the bootstrap, not the transport. Mosh borrows SSH's authentication once, at the start, to deliver a single shared secret — then gets out of the way. That is why Mosh adds no new attack surface to reason about, and why the session survives things that would kill the SSH connection that created it.

So the UDP range needs to be reachable between the two machines. On a tailnet this is automatic — WireGuard carries it, and the ports are exposed to your tailnet only, never to the internet. This is a genuinely important detail: Mosh over Tailscale requires no public UDP ports at all.

Blink Shell

Blink is a terminal for iOS built around Mosh rather than bolting it on. Two properties make it the right choice for this setup.

Keys in the Secure Enclave

The Secure Enclave is a separate coprocessor with its own memory. A key generated there cannot be read back — not by Blink, not by iOS, not by you, not by malware with full device access. The key never exists as bytes anywhere the CPU can see.

Signing works by asking the enclave to sign on your behalf, gated by Face ID or your passcode. Compare that to a passphrase-protected key file, where the decrypted key sits in process memory while you use it, and where a backup of the device carries the encrypted key off with it.

What this buys you concretely

A stolen, unlocked phone with a file-based key is a compromised key: the attacker needs only the passphrase. A stolen phone with a Secure Enclave key is a compromised device — revoke it in the tailnet and remove one line from authorized_keys, and there is nothing to rotate elsewhere, because the key was never extractable and never existed on another machine.

Setting it up

In Blink, open Settings → Keys and create a new key, choosing the Secure Enclave type. Then copy the public key and add it to each server. Blink can install mosh-server for you without root on most platforms: 

# First connection — verify plain SSH works and check the host key
ssh macbook

# Let Blink deploy mosh-server, then use Mosh from then on
mosh --install-static macbook

# The daily command
mosh macbook -- tmux new -A -s phone

Saving a host under Settings → Hosts lets you type mosh macbook rather than a full user/address/key triple, and enables the Files.app integration so you can open remote files in iOS editors.

Trailing -- tmux new -A -s phone

new -A means "attach if the session exists, otherwise create it." One command both starts your day and resumes it, whether the last disconnect was deliberate or the lift doors closing. Make it the saved command on the host and you never think about it again.

Choosing a client honestly

ClientMoshKey storageVerdict
Blink ShellNativeSecure Enclave The recommendation. Built for this exact job.
a-ShellIncludedApp sandbox file Free and genuinely capable; a native shell rather than an emulator, so far more stable than iSH.
TermiusNoKeychain, optional cloud sync Polished, but no Mosh means the drops continue. Cloud key sync is a deliberate trade-off — understand it before enabling.
iSHVia Alpine pkgApp sandbox file An x86 emulator. Fun, educational, and prone to being killed under memory pressure — which is exactly the RST described above.

Checklist

  • Blink Shell installed, with a key generated in the Secure Enclave.
  • That public key added to authorized_keys on each machine, with a clear comment.
  • Plain ssh verified working first, and the host key fingerprint checked.
  • mosh-server present on each machine (mosh --install-static or the package manager).
  • A saved Blink host whose command is tmux new -A -s phone.
  • Verified that Mosh survives switching from Wi-Fi to LTE mid-session.

The commands in this guide change firewall and login settings, and can lock you out of a machine. Practise on something disposable first. Everything here is provided as is, with no warranty — you accept the risk of running it. Read the disclaimer.