Tightly-coupled systems involve two processes running simultaneously and interacting with each other in a dependent way. This involves one side sending data to the other process and vice versa, on a regular cycle. BeamNG can be coupled to a third party (such as MathWorks Simulink) in this manner. This page outlines some considerations which the user should understand when attempting co-simulation with BeamNG.tech.
Tight Coupling:
BeamNG Architecture:
BeamNG operates within a multi-threaded architecture. Graphics rendering, collision detection, UI etc. all execute on separate threads. Also running on its own thread are the physics computations.
Figure 1 shows a simplified overview of this, where the physics steps are shown to be running concurrently with various other processes.
Wall-Clock Time refers to the time which has passed in the real world.
Simulation Time refers to the amount of time which has been simulated.
Note that we can simulate ahead in time, so the current simulation time could be ahead of the current wall-clock time.
BeamNG.tech has both graphics steps (frames) and physics steps:
Frames describe how often the user receives a visual update on screen (ie the refresh rate of the graphics). We usually measure this in frames-per-second (FPS). For example, if we have 30 FPS then we update the image on screen 30 times per second. This quantity is variable, and it is common to see this change significantly when looking frame-to-frame.
Note: BeamNG.tech tries to guarantee the the FPS will not drop below 20 FPS at any time. If this does happen, the physics will go into a slow-motion mode.
At the start of a frame, predictions are made as to when the next frame is expected (in wall-clock time). BeamNG.tech can then easily compute the number of ‘physics steps’ to take the simulation up the predicted start of the next frame (again, in wall-clock time). Execution continues like this.
Physics Steps compute exactly 0.5ms of simulation time. This quantity is fixed at run time, and we often write this as 2000Hz (the simulation frequency). Within the frame, even although BeamNG.tech knows how many physics steps must be computed, we do not know when they will be computed during the frame, although they will typically be computed ahead of the true wall-clock time. There are many things competing for resources and scheduling can shift tasks around as required for optimal simulation. Each physics step can also take a different amount of time to compute.
Although this micro scale appears random and unpredictable, if we take averages over a sufficient amount of time (eg a minute), we will see that there are 2000 physics steps computed per second.
Note: using a command line argument, it is possible to execute BeamNG at different rates eg 4000Hz. However 2000Hz is the supported standard.
In Figure 1, two possible cases are highlighted:
The Left Frame illustrates the case where the physics steps are computed earlier than the work being performed on the other threads. This results in a gap on the physics thread towards the end of the frame. The other threads are the frame bottleneck here, not the physics. This is the more common case. Note the gap in physics steps towards the end of the frame - the physics steps could be said to typically compute in batches with gaps in between.
The Right Frame illustrates the opposite case; where the other threads finish earlier than the physics steps. We say here that the physics steps are the frame bottleneck.
An important point to note here, particularly when looking over multiple frames, is the irregular natre of when actual physics steps occur. We cannot predict when future physics steps will be, since this depends on many non-deterministic factors (especially within Human-In-The-Loop systems). Further to this, depending on what needs to be computed, some physics steps can take longer than others to compute.
Execution:
Timing Measurements:
In order to maximise the efficiency of the co-simulation, the user must provide accurate timing measurements for two things:
The Third Party Computation Time is the time required for the third party (eg Simulink) to compute one iteration of simulation (ie to advance itself by one time step of 0.5ms). If the third party time varies a lot, the user could choose either the maximum or the average time and investigate which provides the most efficient results. Most likely, the best efficiency will be reached when the third party iterations take a regular amount of time to compute.
The UDP Round-Trip Time is the time required for the udp communication infrastructure to send a message from the BeamNG.tech machine to the third party machine, and back again. Even if both ends of the coupling are on the same machine, this should still be provided (although the value will be very small and thus not very significant).
A standard ‘ping test’ from a terminal is sufficient in determining this timing measurement. For example, on Windows systems:
- Type “cmd” to bring up the Command Prompt.
- Open the Command Prompt.
- Type “ping” into the prompt, followed by a space, then the IP address to test with (same IP as the third party machine, or see the .csv file for the default).
These two timing measurements are set in the Available Signals Window, and will appear in the connection section of the .csv file.
V-Sync:
The user should ensure that the V-Sync mode is switched off in BeamNG.tech. If it is switched on, optimal coupling will not be achieved since V-Sync will add extra latency. This option can be found in the Display Menu of the main options screen (press ESC).
Coupling Cases:
In Figure 2, we illustrate the simplest co-simulation case. This is where we have one-for-one data passing between each side, and where the third party (Simulink in this case) has a computation time which is similar in length to the BeamNG.tech physics step computation times.
Here, we say the window size (of the coupled system) is one - for every message we send, we receive one message back. When BeamNG.tech sends a message, the third party is ready to process it, and vice versa.
Note: This simple case is ideal, but it is not typical or even likely. We highlight it here simply for clarity.
Figure 3 illustrates the case where the third party computation time (Simulink) is similar in length to the BeamNG.tech physics step, however, the UDP round-trip time is significantly larger.
For efficient coupling here, we need to have multiple messages sent out before any are received back in BeamNG.tech. Internally, BeamNG.tech will use the two given timing measurements (see above) to compute the optimal coupling management (the window size, etc) which will send, receive and block execution at the most appropriate times, with an aim to provide the most-optimal coupling possible between the two sides.
Figure 4 shows the opposite case; the third party computation time is significantly slower than the BeamNG.tech physics step time, but the UDP round-trip time is fast.
Here, it is optimal to have the coupled system skip sending messages on every second physics steps. We say we have a window size of two, in this case. If messages were sent at every physics step, the the third party would still be processing the previous step and would need to buffer the received messages. This buffering would lead to sync problem in a short amount of time, making the coupling unuseable. The buffering would also grow over time, making the problem worse and using more and more memory. It is not feasible. The clear bottleneck in this case, then, is the third party computation time.
Simulink S-Function:
For MathWorks Simulink, we also provide an S-Function to be used with the BeamNG.tech co-simulation. S-Functions are compiled blocks within Simulink, which handle communication with BeamNG.tech and controls the execution in Simulink appropriately. Figure 5 provides a brief overview of how the S-Function has been implemented (in terms of data flow).
