---

Abstract

Contents

• Objective
• Tasks
• Topicality of the work
• Alleged scientific novelty
• Planned practical results
• 1 Overview of researches and developments
• 1.1 A review of international developments
• 1.2 Review of national developments
• 1.3 Overview of local developments
• 2 Unity Network Protocols
• 3 Researched client-server application architecture review
• 4 Methods and tools for measurement of network protocols effectiveness
• Conclusion
• References

Objective

The goal of this work is to study, select and improve the performance of Unity network transport layer protocols, and, as a consequence, improve the performance of Unity client-server application with intensive network traffic.

Tasks

This work is required to develop a research plan, choose network protocols and platforms to research, explore every single combination and to draw conclusions. Thus, it is possible to form the following sequence of tasks:

1. Review the existing data transfer protocol
2. Review of advanced protocols based on UDP and TCP
3. Wording problems in the form of network protocols performance criterias and methods of measuring metrics
4. Selecting tools for network traffic analysis and measurement of network protocols performance
5. Research of transport layer protocols in Unity [1]
6. Separation of client-server application network traffic via the communication channels using different protocols Unity
7. Efficiency measurement for network operations, traffic analysis in different operating environments and using web sockets
8. Optimization of client-server application based on the received data

Topicality of the work

Optimization of client-server applications has been and remains an actual topic nowadays, especially as the traffic volume grows every year. It is even more actual for gaming applications, where performance of the whole application depends on stability and quality of the traffic.

Alleged scientific novelty

Scientific novelty consists in increase of efficiency of client-server gaming applications based on Unity, as well as in comparing general network characteristics of this technology when using various instruments of delivery of network data.

Planned practical results

It is planned to obtain a specific value of a performance measurement of network protocols with different settings (QoS, platform, web sockets), and then optimize the tested game client-server application. The result should be an increase in the maximum of concurrently active users in the application.

1 Overview of researches and developments

There are two basic network transport layer protocols: UDP and TCP. In that time, the first protocol is unreliable and simple, it has more performance than TCP. TCP, in turn, guarantees delivery of data and its sequence, which is very important.

These protocols were developed decades ago and in some cases don't match requirements. Such problem, for example, raised after increase in the productivity of distributed computing, when the long distance networks required to transmit huge amounts of data – TCP was too slow, and UDP is unreliable [2]. Therefore, there was developed a set of advanced protocols, UDP-based and TCP-based [3,4]. However, the performance of protocols based on UDP advantageously stand out against TCP extensions, so they have received special attention of researchers over the last decade.

1.1 A review of international developments

According to the research [5], the performance of UDP and TCP is affected by not only bandwidth, the size of the socket buffer or the selected size of packets, but also such factors as:

• use of in-kernel features to avoid creating unnecessary temporary copies of memory, and thus, the optimization of CPU utilization time, so there will not be an idle between sending two separate packages
• the use of high-frequency timer to control packets sending, this can avoid mass simultaneous emissions (important for the huge amount of data and big bandwidth, e.g. 1 Gbit/s)
• the use of multi-threading, which can either to speed up the process of data exchange or to slow down it, as the synchronization process is added
• the algorithm of aknowledge about arrival of the package or set of packages
• the algorithm of handling the situation when the package was lost

Given this factors, there were developed many network protocols based on the UDP transport layer protocol, as well as some enhancements to the TCP protocol. Some representatives of these latest protocols described below.

RBUDP (Reliable Blast UDP) – high-speed protocol for transferring large amounts of data, it is an important part of many scientific applications such as distributed computing that actively use datasets. The protocol uses an aggressive algorithm of massive blasting network packages and is intended only for networks with an extremely large bandwidth, or a dedicated fiber-optic networks.

Tsunami – the UDP-based protocol for high-speed file transfers over networks that have high bandwidth-delay product. This protocol is on demand because TCP works pretty poorly in networks with high levels of the metric described above. The protocol divides files into numbered blocks of 32 KB. Communication commands between the client application and the server communicate via low-bandwidth TCP connection while the data is transferred using UDP.

UDP-based Data Transfer Protocol (UDT) – high-performance Protocol for transmission of large data on high-speed long distance networks [8]. Was originally developed and tested on very high speed networks (1Gbps, 10Gbps, etc.), however, the latest version of the protocol was updated to support the conventional Internet. This is one of the most popular solutions for supporting high-speed data transmission and is part of many research projects and commercial products.

Scalable TCP – modification of TCP that is designed to provide much higher throughput and scalability [3]. Scalable TCP modifies the congestion control algorithm. Instead of having to halve the size of the congestion window, each packet loss decreases the congestion window to a small fraction (1/8 times instead of 1/2 in standard TCP), while packet loss will not stop. When packet loss stops, the speed increases at a slow fixed rate (a single packet is added for every hundred successful acknowledges) instead of a standard recovery rate of TCP, where the speed value is inverse of the congestion window size (thus, for very large windows, recovery takes a very long time). This reduces the recovery time on the communication channel 10 GB/s from 4+ hours (using standard TCP) to less than 15 seconds, when RTT is approximately 200 milliseconds.

QUIC (Quick UDP Internet Connections) – a new protocol from Google based on UDP. The protocol is designed primarily to work with HTTP/2, because multiplexing and connection (parts of the HTTP/2 protocol) already implemented in the QUIC. QUIC also has its own encryption layer instead of TLS 1.2. Also there is preventive error correction system included in this protocol – each package on 10% consist of data from other packages, allowing you to reconstruct any lost package according to his neighbors. Chrome has experimental support for the QUIC from 2014, from the backend QUIC supported by such services from Google like Youtube and Google Search. If QUIC will prove its effectiveness, there is a chance that tested ideas will become a part of the next version of TCP [9].

1.2 Review of national developments

In Ukraine, there isn't any development of UDP-based or TCP-based network protocols for common use. However, there is a work related to the use of the TCP/IP stack for telecommunications with the changing size of the network address (EX) [12]. This work was conducted at the Institute of telecommunications and global information space of National Academy of Sciences of Ukraine by senior research fellow Gulyaev Kirill Dmitrievich.

There are also many works related to fundamental properties of the protocols on theoretical level.

In the Kherson National Technical University was made a research – analysis of conformity at the stage of designing network protocols, which proposes to improve the life cycle of a network protocols creation for greater reliability [10].

At the National Technical University of the Kiev Polytechnic Institute there was created an article called The implementation of network protocols as independent processes. The article discusses the approach to implementation of protocols that allows you to implement the logic of each network protocol in the stack completely independent of other protocols[13].

But it is worth noting that the most of scientific papers about network protocols in Ukraine devoted to the studies of the properties of common network protocols and their modifications.

In Kharkiv National University of Air Forces in 2009 were published a work called The particular implementation of the TCP protocol in modern computer networks, which deals with the issues of implementation of modern versions of TCP [11]. And at the National Aviation University was made a study of protocols for routing in Ethernet networks of class Metro [14].

1.3 Overview of local developments

DonNTU masters never have been developed or investigated the network transport layer protocols based on TCP or UDP before, but there are some researches of the implementation of routing protocols and channel layer protocols:

• Kravchuk Vasylii (Research and improvement the protocol of data transmission by networks of electrosupply 220V, 50Hz for SCADA-systems)
• Mikhail Zub (Analysis of Routing Algorithms in Dynamic Networks Based on ZigBee)
• Artem Timkov (Optimization of Wireless Sensor Systems Using Ant Algorithms)

2 Unity network protocols

Tested network application written using a powerful Unity platform, however, it imposes a restriction on the use of other technologies. Therefore, in this work researched only protocols implemented by Unity developers.

Unity implemented a high-performance transport protocol based on UDP, where you can specify different QoS (Quality of Service). QoS is the criterion that should guide the protocol during its work. Unity developers have implemented several types of the QoS parameter [1]:

• Unreliable – untrustworthy messaging channel, where messages may be lost due to network problems or an internal buffer overflow, similar to a regular UDP packet. Communication channel with such QoS type can be used for logging non-critical processes in the application.
• UnreliableFragmented – максимальная длина пакета неизменна, но этот тип канала перед отправлением будет разбирать длинные сообщения на фрагменты и собирать их обратно перед получением. Так как такое качество обслуживания ненадежно, доставка не гарантируется. Пример применения: логирование длинных записей в журнал. maximum packet length is constant, but this type of channel disassemble a long message into fragments before sending and collect them back before receiving. This quality of service is unreliable, so delivery is not guaranteed. Application example: logging long messages.
• UnreliableSequenced – the channel ensures the order of delivery, but since this quality of service is unreliable, the message may be lost. This type of channel can be used for voice or video messages, allowing high performance.
• Reliable – the channel guarantees delivery (or disconnection), but does not guarantee the order. Application example: data transfer of one-time events in the game, such as damage taken by the player.
• ReliableFragmented – same that UnreliableFragmented, but in addition guarantees delivery. Can be used in cases when you need a reliable data channel where the order is not important and the packages are large.
• ReliableSequenced – same that UnreliableSequenced, but guarantees delivery. This QoS is similar to the TCP stream. Application example: transfer files and patches.
• StateUpdate – ненадежный тип канала, принудительно сбрасывающий пакеты старше получаемого/отправляемого. Если при передаче буфер содержит более одного сообщения, отправлено будет только самое свежее. Если буфер получателя при чтении содержит более одного сообщения, только самое свежее будет доставлено. Пример применения: передача расположения. unreliable channel type, forcing to drop the packets that over last received/sent. If the transfer buffer contains more than one message only the freshest one will be sent. If the buffer of the recipient when reading contains more than one message, only the freshest one will be delivered. Application example: transfer of location.
• ReliableStateUpdate – same that StateUpdate, but additionally guarantees delivery.
• AllCostDelivery – very similar to Reliable, but there are differences. The Reliable channel will re-send the message based on the transit time of the signal in both directions (round trip time value (RTT)), which is determined dynamically, while AllCostDelivery will automatically forward messages after a certain period of time (set in settings). This can be useful for small but important messages.

In order to use these channels you need to create them using the API or directly in the Unity visual editor, using built-in NetworkManager component as shown in Figure 1.

Figure 1 – Connection channels settings of NetworkManager component

Channels are automatically assigned with indexes starting from zero, which you can then use in your code to specify which channels should be used to send particular data.

Data about the position and moving of any entity that have position in space is transmitted using a built-in NetworkTranform component. For the remaining data (for each variable in the program code) so-called SyncVar synchronization attributes are created. The values of these variables are synchronized between server and clients. This data is sent on channel 0 (zero) by default, which defaults to QoS Reliable Sequenced (which is practically equivalent to sending via TCP).

However, these channels can be changed at the level of script components using NetworkSettings attribute [7]. This attribute takes two parameters - a channel number and the synchronization interval. It is worth considering that interval in this case only as a limiter. States of variable with SyncVar attribute transferring over the network only at the moment when it was changed. If such a variable changes very often, for example, at each frame of rendering a game scene, this can lead to overflow of the communication channel. Therefore, system waits this time interval since the variable changed and only then the last state of this variable is transferred over the network. The standard interval-limiter is equal to 100 ms. Example code of a simple Unity script component which sync data across the network shown below in Figure 2.

using UnityEngine.Networking;

[NetworkSettings(channel = 1, sendInterval = 0.2f)]
class MyScript : NetworkBehaviour
{
    [SyncVar]
    int value;
}
					

Figure 2 – Code of Unity script component with NetworkSettings attribute which contains a variable with SyncVar attribute

In order to evaluate the performance of each protocol in Unity there will be several tests. The first test is replacing the QoS on the main, zero channel. Thus, all network traffic of the game client-server applications will go through this channel. Then, for each component of a client-server application is chosen the channel with the most suitable QoS. This should give a noticeable increase in performance.

This set of tests will also be runned on multiple platforms - Windows, Linux and WebGL (using web sockets). Application functioning test on Linux and WebGL will be runned both locally and remotely, deploying the application on real dedicated server (VPS) and web server (hosting).

3 Researched client-server application architecture review

Tested client-server application is a game application based on Unity technologies (which provides good cross-platform abilities) and will be deployed as a desktop application for PC (Windows, Linux) and browser-based WebGL app.

The server part of the application is situated on a remote server, which operates in operational system CentOS 7. The app uses a technology called Master Server Barebones, which helps for the rapid creation of infrastructure for functioning of this online application. The server part consists of three modules: the Master Server, Spawner and Host.

The Master Server is responsible for user registration, holding a user profile, registering created game rooms (lobbies) and transmitting requests for creating gaming sessions from users to Spawners.

When the game room had accumulated a sufficient number of people and the room creator starts the game session, Master Server creates a task to run Host. This task go in the queue on the Master Server. Master Server filters all registered Spawners by region and availability, and if Spawner that meets the requirements is not busy Master Server passes him the task to run Host.

Thus, each physical/virtual server must have one Spawner – the component responsible for creating processes of the Host. After creating a task, Spawner launches a new copy of the Host process, and IP and port number of the Host is transferred through the Master Server to all players in the room. After that players are connected to new Host.

The Host in this case is a reduced copy of the main application, which essentially is a mini-game server for an individual game session. It loads the main game scene and all data is transmitted through the Host to all connected players. The Host plays the role of a so-called relay server that allows to connect players which don't have a dedicated IP and are in different network segments with firewalls and routers with NAT.

This network architecture in the dynamics shown in the animation below. The animation shows the entire process of login, creating a game room, starting the Host and connecting of two players to it. The study should optimize the flow of data between the Host (also called a Game Server) and connected players.

Figure 3 – The functioning of the investigated client-server application network architecture in dynamics (animation: 15 frames, 5 loops, 173 KB)

It is also worth noting that the application uses 3 databases.

The first database is a simplified database with accounts and profiles on the Master Server. Barebones Master Server uses LiteDB by default – this technology is a lightweight file-based DBMS.

The second database contains detailed information about accounts and created game maps, which are tied to the account. It is located in MLab cloud, using NoSQL MongoDB database. The advantages of this technology is that the record size can vary greatly depending on the size of the stored game map, that allows to save space on the harddrive. The disadvantage of this engine is that it is using RESTful API instead of SQL. Thus, the complex logical operations and filtering of the data must happen in the application after receipt of all raw data.

For this reason third database with traditional MySQL was deployed on available web hosting. This database collects player stats, battle stats and information about the resources of the account. Further, these data will be actively used in various SQL queries to gather game statistics.

The current distributed data storage architecture has two shortcomings.

The first disadvantage is redundant information about accounts, which is partially duplicated in LiteDB and MongoDB. Thus, to enter the game application the users wait until they pass successfully both requests for authorization. This also raises questions about information security. This disadvantage arose due to lack of time resources and can be eliminated by implementing an adapter class for MongoDB (the framework provides such availability).

The second flaw, which is visible to the user – it is a slow retrieving and writing data to MongoDB. For security purposes, on the side of the web hosting has been written intermediate API to access the database on the MLab cloud service. The application communicates with the database of accounts and maps exclusively through this API. Moreover, the speed slows down even more due to the fact that there is free test plan selected on the MLab service (so-called sandbox), which is limited in speed. All this leads to long logging in the game and downloading game map when using map editor. To eliminate this drawback, transferring to a private dedicated server with MongoDB DBMS is considered.

4 Methods and tools for measurement of network protocols effectiveness

Let's review a few tools to measure the performance of network protocols . The first of them is a built-in Unity profiler, which can measure not only the performance of CPU and GPU, but also the number of network operations. The window of the Profiler is shown below in Figure 4.

Figure 4 – Unity network profiler

As can be seen in the figure, the profiler calculates characteristics such as the number of network transactions and the number of network messages in a particular frame of the game. These operations are divided into categories, which are used in Unity. These characteristics can be helpful in optimizing the number of generated network requests in the Unity application, but in the context of network protocols research this tool is useless.

Another tool for analyzing network traffic is a Wireshark utility. This program belongs to the category of so-called sniffers and saves all network traffic on the fly. Later in the saved network traffic you can view information of the data packets per each byte. In order to filter out unwanted traffic and show only relevant packets (e.g. coming from one IP address to another), Wireshark has built-in powerful system of filters, examples of which can be found in [15].

However, with all power of Wireshark there is a small drawback – it is the lack of extensive graphical presentation of statistics. Menu Statistics provides a general overview of the set of captured data, mainly in tabular form. Although, there are still some charts available, such as Graph Input/Output (I/O Graph). This chart is completely editable, you can add obtained elements using the same powerful filtration system of intercepted data stream. An example of such chart which displays statistics of UDP packets is presented below on Figure 5.

Figure 5 – I/O Graph in Wireshark

There are additional ways of presenting the data captured using Wireshark: it's tools that extend the graphics capabilities of Wireshark (however, many of them are paid), and any other tools that visualize data in CSV format. Wireshark can export all the data in CVS file, which you can later open in another application like Microsoft Excel.

Alternatively,the program called Colasoft Capse Free was reviewed. This program represents a lot of information in a graphical format and offers easy and intuitive interface. Screenshot of this program is presented below in Figure 6.

Figure 6 – Colasoft Capse Free GUI

However, the free version of this program is limited. For example, the maximum buffer size for stored packets is only 8Mb, which is very small for the studied client-server application that intensively uses network traffic. So the choice fell on Wireshark as the main research tool.

Now let's consider what data to collect and explore. In order to evaluate the performance of Unity network protocols, following data will be collected and analyzed:

• bandwidth of the network;
• amount of traffic per unit time;
• average packet size;
• maximum packet size;
• average delay;
• maximum delay;
• packet loss level at different load levels;
• total number of all transport layer protocol errors.

The biggest criterion of effectiveness should be the metric of data transfer speed. It is possible to obtain it simply by dividing the average packet size on average delay. The quality of communication can be judged by the level of network errors, which are automatically tracked by Wireshark.

As has been written in this article before, to evaluate the effectiveness of each QoS in Unity a series of tests will be run, with a change in QoS on the communication channels, as well as changes of data transmitted through them.

This set of tests will be run on such platforms as Windows, Linux and WebGL (using web sockets). When testing in Windows there is an ability to run locally both client and server parts of the application, and that fact should make data collection easier. In Linux, the most important object of testing is the server part that will be tested with different workload on the app. For testing web sockets on the WebGL platform, backend (which still will work on Linux) have to be recompiled. Meanwhile, the WebGL client part will be located on the web-site and run through any modern browser.

Conclusion

Review of the scientific literature and developments about creation and researching of the advanced network protocols effectiveness showed that this topic will always be relevant, because increased performance of networks at the channel layer requires the creation and modification of protocols at the transport layer.

Analysis of documentation and other popular sources about UNet network technology in Unity shows that the effectiveness of the protocols in UNet are not fully covered even by the Unity developers thrmselves, and the partitioning of network traffic between communication channels used only by advanced game developers.

Thus, there is a need to run a large series of tests described above using appropriate tools for network traffic analysis, and then compare results and draw conclusions.

References

1. UNET — новая сетевая технология в Unity 3D // Хабрахабр [Electronic source]. — Access Mode: https://habrahabr.ru/post/228395/
2. Eric He, Jason Leigh, Oliver Yu, Thomas A. DeFanti. Reliable Blast UDP : predictable high performance bulk data transfer // ResearchGate [Electronic source]. — Access Mode: https://www.researchgate.net/publication/3985811_Reliable_Blast_UDP_predictable_high_performance_bulk_data_transfer
3. Tom Kelly. Scalable TCP: Improving Performance in Highspeed Wide Area Networks // CERN [Electronic source]. — Access Mode: http://datatag.web.cern.ch/datatag/papers/pfldnet2003-ctk.pdf
4. David X. Wei, Cheng Jin, Steven H. Low, Sanjay Hegde. FAST TCP: Motivation, Architecture, Algorithms, Performance // NetLab CalTech [Electronic source]. — Access Mode: http://netlab.caltech.edu/publications/FAST-ToN-final-060209-2007.pdf
5. Yunhong Gu, Robert L.Grossman. Optimizing UDP-based Protocol Implementations // SourceForge [Electronic source]. — Access Mode: http://udt.sourceforge.net/doc/pfldnet2005-v8.pdf
6. Sean Riley. UNET SyncVar // Unity Blogs [Electronic source]. — Access Mode: https://blogs.unity3d.com/ru/2014/05/29/unet-syncvar/
7. NetworkSettingsAttribute // Unity Documentation (Version: 2017.2) [Electronic source]. — Access Mode: https://docs.unity3d.com/ScriptReference/Networking.NetworkSettingsAttribute.html
8. UDT: Breaking the Data Transfer Bottleneck // SourceForge [Electronic source]. — Access Mode: http://udt.sourceforge.net/
9. Протокол QUIC: переход Web от TCP к UDP // Хабрахабр [Electronic source]. — Access Mode: https://habrahabr.ru/company/infopulse/blog/315172/
10. Изотов А.С., Немченко В.П., Анализ конформности на этапе проектирования сетевых протоколов // КНТУ [Electronic source]. — Access Mode: http://epr.kntu.net.ua/116/1/34.pdf
11. А. В. Карпухин. Особенности реализации протокола TCP в современных компьютерных сетях. Системи обробки інформації. — 2009. — № 6(80). — С. 49-53.
12. Гуляев К.Д. Реалізація прикладного програмного забезпечення, що використовує телекомунікаційну технологію із змінним розміром мережної адреси // ХАИ [Electronic source]. — Access Mode: https://www.khai.edu/csp/nauchportal/Arhiv/REKS/2012/REKS412/Gulyayev.pdf
13. Орлова М.М., Шандиба Д.В. Реалізація мережевих протоколів як незалежних процесів // IT-Вестник КПИ [Electronic source]. — Access Mode: http://it-visnyk.kpi.ua/wp-content/uploads/2011/03/51_19.pdf
14. В. Клобуков, В. Рябоконь, С. Єрмак. Дослідження протоколів побудови маршрутів в мережах Metro Ethernet // НАУ [Electronic source]. — Access Mode: http://avia.nau.edu.ua/doc/2011/4/avia2011_4_6.pdf
15. Как использовать возможности фильтров отображения Wireshark по максимуму // Журнал Хакер [Electronic source]. — Access Mode: https://xakep.ru/2013/11/05/wireshark-filtres/