October network logic basically working
Posted by Jeff Disher
October network logic basically working
The network layer logic and framework for October seems to be generally working. This hasn't been stress-tested, nor are all the network packet types implemented (those will be added as needed) but the basic design is in place and working.

This is a pretty basic single-threaded asynchronous event loop. Since it is only receiving incoming connections, this should work well until some kind of CPU-heavy compression mechanism is required (which may never happen as the heaviest part of the data is likely binary without many patterns). Some of the lock-step buffer management can be made more sophisticated, if required.

This should cover most of what is required for basic mechanics (loading/generating a world and walking around), meaning that the next step is probably an initial cut at "bringing it all together" to try to make a complete test of loading and walking through a world. Note that this initial cut will NOT include graphics - although that can be the next integration point.

This means building a basic cuboid generator (still a flat world) and a test to allow multiple client to enter and walk around the world.

The initial challenge here will be this connection and scheduling the communication to the various clients, given that both the network and core logic loop are internally asynchronous.

Another problem which will need to be solved (either as part of this or the immediate next step) is how clients will maintain a "shadow copy" of the state, with speculative mutations applied to it. Figuring this out is essential for making sure that the clients can interact in a coherent way while the state of the system is perpetually out of sync. It will require something like an operational transform although my plan is to make something simpler: Reversible mutations.

I am not sure how good an idea this is but the this would mean a new message from the server, arriving on the client, would result in: The client reverses its speculative mutations, applies the new messages from the server to its shadow state, prunes any speculative mutations older than the server-state local version, and re-applies the remaining speculative mutations, dropping them if they fail to apply on the new state.

This idea may not work but is at least simple, in concept. This should allow good generalizability (especially if the speculative mutations can be stateful to the point where the list of them is essentially an undo stack), even without the updatability performance of something like an operational transform (or, at least as I understand them). The ideas should be largely equivalent, though.

Where things will get interesting is once these fundamentals of the connection are built and I can start testing things like what happens when the clients race to place a block or how do they deal with other inconsistencies/impossibilities (walking through a wall someone just placed, for example - some heuristics will be required, unfortunately).

Of course, there will still be other bugs and corner-cases to figure out, as this progresses (I realized there is already a missing case in the path-finder).

Here's hoping there is smooth sailing in these next few steps,
Jeff.