Hi everyone! Despite being a successful games programmer for a few years now, this is the first time I decided to post here on forum 🙂 So welcome!
I am building a multiplayer game right now that involves cars, that are simulated physically. So I have been doing various networking for years now, and I am very familiar with the client/server architecture, client side prediction, lag compensation and server reconciliation techniques. However I mostly worked on netcode for shooter games, which was a bit simpler than the physics simulation I am dealing with right now.
My technology
I am using Unreal Engine, but that is completely irrelevant as I have integrated separate physics engine(the newest NVidia PhysX) to be able to manually tick the physics engine in order to replay physics and deal generally with the fixed time step. So I have complete control over the physics for my vehicles and the entire scene, and Unreal doesn’t interfere, I made sure of it.
My solution allows me to change time steps at edit time, so I am able to test various fixed time steps without any trouble.
My networking solution
I am using standard networking solution that has been introduced during multiple GDC talks such as Overwatch, Rocket League, Halo and Glenn Fiedler’s networked physics talk.
So, my clients are always running ahead of the server by half RTT + 1 buffer frame. Clients are using upstream throttle in order to simulate faster or slower, which is dictated by the server, to always have some inputs buffer on the server side. This solution is very robust as I have shipped multiple FPP and TPP shooter games without any trouble. The clients only send a list of inputs to the server, from oldest not-yet-acknowledged input, to the current one, so packet loss is covered very well(standard technique).
For the characters it’s all simple, local client is using its own inputs to predict local client and buffers the states and inputs locally, so when server returns state, it checks if correction is needed, and if so, it replays all the frames from the corrected one up until current one to be back on the lockstep. It all works perfectly, I wanted you to be aware I am very familiar with this networking model.
Now to the point…
The regular player characters can be rolled back to the correction frame and replayed extremely easily, because they do not have any physics, they are kinematic beings and therefore their “physics” replays are very, very cheap.
I started implemented my vehicle system, which is deterministic enough(except for the floating point errors, but they are well covered using the corrections, so this can be completely skipped). Each input for the vehicle on all clients and server gives exactly the same results, which is very big win for me. However….
My game is heavily focused on cars collisions, and that’s where the trouble start!
Predicting everything(like Rocket League)
Simulation running at 60hz
In this solution I am predicting local player, which is easy, due to the deterministic nature of the vehicle and physics system I have. There are never ever mispredictions, even when hitting some static objects around.
However, OTHER CARS are not on the same timeline for me as I am. I used the Rocket League solution, where I would predict remote clients just as I predict mine. In this case when I get information about the remote client from the past, I check if it needs to be corrected. If so, I do the replay from that corrected frame to the current one. Standard technique and it works very well. There are some mispredictions pretty often, because I cannot predict remote client‘s input, so all I have is their last used input. If they have changed the direction during that time, I have no way to know that, but that’s okay, the correction and replay covers that well enough, so I don’t really see that.
This technique seems perfect, but… performance. I measured my simulation and 1 physics frame takes usually around 0.5ms – 1ms time, which is very quick, really nice. However. Let’s say I am on the frame 100, but because I am high ping client(let’s say 200ms ping), I may receive information about remote clients from around 200 or more milliseconds ago, which with 60Hz simulation translates to around 13 frames. This is where the trouble starts. If my simulation usually takes around 1ms, I need to replay the simulation from 13 frames ago, up to current one, which is 13ms. This IS SIGNIFICANT. Considering, that the game rendering at 60FPS(16ms), this is over 80% of the time to be added! Of course, we can try to budget the game in a way that it is able to run at 120fps, when we would have enough budget to keep it there, but I am wondering if I am missing something? Or is it really the way it needs to be? Considering how remote clients go out of sync as soon as they change their inputs, this can very often simulate 13 frames of physics during one fixed time step tick. I do not think I can squeeze the physics to simulate faster than that 0.5 to 1ms, this is pretty good as it is.
So do you think I am missing something here? Or is this they way to go, and we should consider optimizing other parts of the game to account for that extra 13ms? Thanks for any responses!
