Discrete events travel on one reliable, ordered lane per connection and direction

Status: Accepted (2026-06-17)

Context

Continuous state (PlayerSync and the other frames) is unreliable and latest-wins: a lost frame is replaced by the next one. Discrete, state-changing events are different: a lost chat line, death or vehicle entry is simply gone, and two events applied out of order can leave the receiver in a state the sender never had. Such events need delivery that survives loss, removes duplicates and keeps order, without slowing down the state stream and without rewriting every packet of the protocol.

This builds on the envelope of ADR 0004: the session token on Client → Server packets and the per-player generation on Server → Client packets.

Decision

• One reliable lane, ordered. Each connection has one reliable stream per direction, and every reliable packet of that direction shares it. Continuous state stays off the lane and keeps its own unreliable, latest-wins path, so resends never delay a frame.
• Reliability is local to the packets that need it. A packet is on the lane when its layout carries event_id: u32; two small packets carry the acknowledgements: ClientAck for the Server → Client stream and ServerAck for the Client → Server stream. Other packets keep their layouts. There is no channel id on the wire while there is one stream per direction.
• Go-Back-N with a cumulative ack. The receiver delivers only the next expected event_id and discards duplicates and events ahead of a gap. Every reliable packet it receives is answered with the highest event_id delivered without a gap, so a lost ack repairs itself and a gap makes the sender resend from the gap. Ordering and deduplication fall out of the scheme; there is no reorder buffer and no selective ack.
• Both hops are reliable. A client resends its events until the server acks them, and the server keeps a separate queue for each recipient and resends to each until that recipient acks. A broadcast event is therefore delivered end to end, not only to the server.
• Fixed retransmission timeout. An event is resent once 250 ms have passed since its last send. The server wakes up for resends every 100 ms; the interval is only the granularity, the event's age decides. Per connection and wakeup it resends at most 32 due events, oldest first: a lost event makes every later one due, so the budget has to outrun the rate at which a busy lane fills.
• No give-up per event. An event is never abandoned while its connection lives; giving up after a number of retries would turn a live connection's "reliable" event into a silent loss. A dead peer is removed by the disconnect rules, and its queues die with it.
• Bounded queues on the server. With 256 unacked events on a connection, new droppable events (such as chat, skills, pickups and wanted level) are not queued for that recipient. Events whose loss would break the game state, such as lifecycle edges (ADR 0017), are still queued up to 512, and reaching that limit disconnects the client. A connection whose unacked events have waited 15 s without an ack moving the queue is disconnected too, even if other packets keep it alive. A refused event never consumes an event_id, so the stream has no gaps.
• A fresh stream per connection. event_id starts at 1 on every connection. Events carry no generation tag: stale Client → Server packets are dropped by the session token before they reach the lane, and a reconnect or a reused player_id gets new endpoints on both sides.
• The client accepts Server → Client packets only from the server's address. Those packets carry no token, so a forged event or ack from another address could otherwise advance the client's stream or clear its queue.

The wire rules are in Reliable event lane; the server endpoint is src/reliable.rs.

Consequences

• Every new discrete event joins the existing lane by adding event_id to its packet; no new transport is needed.
• One stream per direction means head-of-line blocking: a lost event delays every later event of that connection until it is resent. For the event rates of the game this costs at most a few hundred milliseconds under loss.
• Out-of-order events are resent rather than buffered, which wastes some bandwidth under loss and keeps both endpoints simple.
• Still open, for when a real consumer needs them: separate unreliable-sequenced, request and bulk lanes, a channel id on the wire, selective acks, an RTO adapted to the round trip, and a rate-limit policy for high-rate events.

Alternatives considered

• A generic envelope with lane and sequence fields on every packet. It would rewrite the whole protocol and every fixture to serve the few packets that need reliability. Rejected.
• Five lanes up front (state, unreliable events, reliable events, requests, bulk). Only reliable events had a consumer; the others would be design ahead of need. Rejected.
• Selective acks and a reorder buffer. At the event rates involved the extra resends of Go-Back-N are negligible. Rejected as premature.
• An adaptive RTO. Premature without a measured problem with the fixed one.
• Bounded retries, then drop. Silent loss on a live connection. Rejected.
• A generation tag on every event. It would duplicate what the session token and per-connection endpoints already guarantee. Rejected.