O3DE / AWS server + client networking for multiplayer host

Online multiplayer games have a long list of requirements to provide the best possible experience for players: game clients need identities that can securely access the game backend, backend services are required to host functionalities like matchmaking and player data, and you need a scalable way to host game sessions.

In this post you’ll learn how to leverage Amazon GameLift, a dedicated game server hosting solution, to host C++ game servers with ease. Learn how to build a fully serverless backend for your game, and use the AWS SDK for C++ to securely communicate between the game client and backend. Whether you’re using a proprietary C++ game engine, or a C++ based commercial engine, this solution can help accelerate and remove the heavy lifting in multiplayer game development, enabling you to focus on the fun stuff.

The full solution for game server, game client and backend functionalities, with source code, and deployment instructions, can be found in the AWS Sample GitHub. This game sample covers use cases for both C++ game servers and the client, as well as a Unity game engine-based server and client. This post will focus on the C++ setup, but you can apply the same AWS architecture to any of the above options.

Before we run through detailed deployment steps, here is an overview of the components of the solution.

The serverless backend
Building game servers isn’t as simple as deploying and connecting to them from the client. A backend solution is required to manage player identities, player data, and matchmaking requests. But building and managing this infrastructure can be a full time job. By leveraging AWS serverless services, you can easily scale your game to player demand, without having to worry about operating and managing infrastructure. For this solution, we utilize a serverless backend architecture built from the following AWS services:

Amazon Cognito is used for identity pools to host player identities.
Amazon DynamoDB is used to store player data and matchmaking tickets.
Amazon Simple Notification Service (SNS) will receive matchmaking events from Amazon GameLift FlexMatch.
Amazon API Gateway hosts API endpoints.
AWS Lambda is used to host functions to communicate with Amazon GameLift for matchmaking, and Amazon DynamoDB for player data.
architecture diagram of GameLift, Serverless C++ game server solution

Game server
Amazon GameLift supports Windows Server 2012, Amazon Linux, and Amazon Linux 2. For this example, I am using Amazon Linux 2 for maximum performance and fast scaling. Running a headless version of your game on Linux as your game server is common practice. This lets you run the same simulation on the server side as you do on the client, and utilize your selected networking solution to manage client-server communication.

This example uses a C++ game server implementation that integrates with Amazon GameLift. Automation is used to download and build the GameLift C++ Server SDK, and build and deploy the server to Amazon GameLift. The solution also demonstrates how to integrate the GameLift C++ Server SDK with your server code.

The game server works with GameLift Local, which you can use to test your integration. Learn more about testing integrations with this documentation guide. It also implements a minimalistic blocking TCP server that accepts two clients, one at a time, and validates the player session IDs received from the client. This helps you to ensure that the correct players are connected to the server. However, the purpose of this example isn’t to demonstrate a TCP client-server networking architecture, and so you will need to replace the socket implementation with a proper asynchronous design. This is commonly implemented with libraries such as boost.asio or higher level game networking libraries such as ones provided by your game engine.

This solution uses Amazon CloudWatch Logs to collect the game server logs. The game server writes output via the log files defined in the CloudWatch Agent configuration which is done as part of the fleet deployment. This allows the CloudWatch agent, which is installed on the Fleet instances, to push logs for the game servers running on every instance independently.

GameLift resources
This example deploys the following Amazon GameLift resources to host the game servers and matchmaking functionality:

FlexMatch Matchmaking rule set defines a single team with 4 to 10 players and a requirement for the player skill levels to be within a distance of 10 from each other. In the example, all players have the same skill level, and player skill is stored in the backend service using DynamoDB. There is also an expansion configuration to relax these rules to a minimum of 2 players after 5 seconds. When you connect with 2 clients, you will see this 5-second delay before the expansion is activated. The FlexMatch rule set also defines a latency requirement of < 50 ms for the clients. This is relaxed to 200 ms after 10 seconds.
FlexMatch matchmaking configuration uses the rule set and routes game session placement requests to the queue.
GameLift Queue places game sessions on the GameLift Fleet. In the example, there is a single fleet behind the queue, and it has two regional locations (home region and one secondary region). You could have multiple fleets within the home region (for example a Spot Fleet and a failover On-Demand Fleet for cost optimization).
GameLift Fleet sits behind the queue and utilizes the latest game server build. The fleet has two regional locations and runs on Amazon Linux 2. Each instance runs two game server processes, but you can pack more game servers on each instance based on the instance size and the resource requirements of your game server. Our example utilizes the c5.large instance type, which is a good starting point for compute intensive workloads such as game servers.
architecture diagram of GameLift game servers and matchmaking

Game client
The game client is a simple Linux C++ client that communicates with the backend and the game server. This example uses the AWS SDK for C++ to request a Cognito identity, and to sign in and make requests against API Gateway. Your game client is likely built on Windows, but you can also leverage the example on Windows by following the instructions in the build and use the AWS C++ SDK on Windows developer guide. You can also find information in the How to build and integrate the AWS SDK for C++ games on Windows blog post.

Deploying the solution
Now that you understand on a high level what the components are, let’s deploy the solution!

This example uses AWS Cloud9, a cloud-based integrated development environment (IDE), to deploy every resource and build the C++ game server and client. AWS Cloud9 allows you to run an isolated environment on Amazon Linux 2 directly in the browser. It will have your AWS access credentials automatically configured and many of the required tools pre-installed. By utilizing the same Linux version as Amazon GameLift provides a great testing environment for the game server.

To create the AWS Cloud9 environment, open the AWS Management Console, navigate to AWS Cloud9 and select Create Environment.

AWS Cloud9 console page

Then, set the name and description of the environment to reflect what you’re working on, and select Next step.

AWS Cloud9 create environment page

Leave most of the configuration to the default values, but select m5.large for the instance type, as you are using relatively heavy processes to build the different SDKs. Then select Next step.

AWS Cloud9 create environment page

Review the configuration and select Create environment. This will open up the Cloud9 environment which will start in a few minutes.

AWS Cloud9 create environment page

Resize the volume to fit everything required. Open another tab, navigate to AWS Cloud9, and select the instance name to open the details. You can also do this by selecting View details.

AWS Cloud9 instance details

Select Go To Instance to open the instance in the Amazon EC2 management console.

AWS Cloud9 instance details

Select the instance checkbox, then select the storage tab, and select on the volume ID to open the volume.

instance details storage tab

Select Actions and then Modify Volume to change the volume size.

changing volume size

Set the size value to 40 GiB and select Modify.

Modify volume

Then, Click Yes to modify the volume size.

Reboot the instance for volume resizing to take effect in Cloud9. Navigate back to the instance in the EC2 Management Console, select Instance state, and select Reboot instance.

Reboot instance

Click Reboot to reboot the instance.

Now, go back to the browser tab that has the AWS Cloud9 IDE open, and wait for the instance to reboot.

AWS Cloud9 development environment

The Cloud9 development environment should now be up and running. One more requirement for the setup is cloning the repository to the Cloud9 environment. Go to the terminal in the Cloud9 environment and run the following commands:

git clone github.com/aws-samples/aws-gamelift-and-serverless-backend-sample.git

cd aws-gamelift-and-serverless-backend-sample/

This will clone the repository using the preinstalled git client and open the root folder. You are now ready to deploy the game backend.

Serverless backend deployment
Now that you have the development environment up and running, the next step is to deploy the serverless backend.

The infrastructure is deployed with the Serverless Application Model (SAM), an open-source framework for building serverless applications. SAM utilizes a YAML template to define the resources, which can be found in /GameServerAPI/template.yaml in the repository. This template defines every resource, including DynamoDB tables, SNS Topic, API Gateway, Lambda functions, and the IAM access policies for the Lambda functions.

A key configuration in the template is to set DefaultAuthorizer: AWS_IAM for the API. This requires the client to sign requests with credentials provided by Amazon Cognito to validate their identity.

Open GameServiceAPI/deploy.sh in the Cloud9 editor by navigating to the file in the Cloud9 file browser on the left and double clicking it. You could now set the preferred AWS region as the value of region for the backend resources, but leave it to “us-east-1” default to simplify the deployment.

Set a globally unique name for the Amazon S3 deployment bucket. This is utilized by SAM to deploy the resources. The script will create the bucket, build the SAM application, and deploy it.

Deploying SAM application

Then, run the following command in the Cloud9 terminal in order to deploy the backend:

cd GameServiceAPI && sh deploy.sh && cd ..

This will build and deploy the components in the earlier architecture diagram, excluding Amazon Cognito Identity Pool which you’ll deploy later. The deployment can be followed from the terminal and from AWS CloudFormation console. Once completed, the AWS Lambda console is a great place to review the resources of a serverless application.

Navigate to the AWS Lambda Console, select Applications, and select gameservice-backend to review the backend application, which shows the API endpoint and has links to every deployed resource.

AWS Lambda console

Now you will deploy some additional pre-requirements: the Amazon Cognito Identity Pool with related roles that allow access to the backend API, and an IAM Role for the GameLift servers. The IAM Role for the servers are required for configuring the Amazon CloudWatch agent to send logs to CloudWatch, so you need to create it before the GameLift Fleet. You can review FleetDeployment/prerequirements.yaml for details regarding the pre-requirements resources.

Run the following command in the Cloud9 terminal in order to deploy the pre-requirements (you will use the default “us-east-1” region so no changes are needed in the script):

cd FleetDeployment && sh deployPreRequirements.sh && cd ..

Once that has completed, every backend resource should now be deployed, and you are now ready to build and deploy the server.

Server and GameLift resources deployment
The game server consists of the following code files:

CppServerAndClient/Server/Server.h: Declares the server class and all the related GameLift callbacks.
CppServerAndClient/Server/Server.cpp: Implements the server class and contains the entry point of the application.
In the main() function in Server.cpp, we call InitializeGameLift with the appropriate port and logfile configuration. The port is received as a command line argument from the GameLift Fleet, and the logfile is configured based on the port. GameLift is initialized by calling Aws::GameLift::Server::InitSDK(). The callbacks for different GameLift events, logfile, and port are configured with Aws::GameLift::Server::ProcessReady(). The main() function of Server.cpp will then setup a simple TCP server to accept two player connections.

Once a game session is created and you receive a successful connection from a client, the server will validate the GameLift player session ID received from the client by utilizing the GameLift Server SDK. Using this logic, drop any clients that don’t have a properly created session in order to increase security.

After receiving two connections and player session IDs, the server will wait 10 seconds, and then terminate GameLift properly by calling Aws::GameLift::Server::ProcessEnding() before terminating the process. This lets GameLift automatically and immediately replace the process with a new one. It is best practice to use your game server processes for just a single game session in order to avoid any cleanup or memory leak issues.

Before building the server, first configure the IAM Role created earlier for GameLift to be utilized by the CloudWatch agent on the fleet instances. This allows the instances to have the required permissions to write to CloudWatch Logs.

Open CppServerAndClient/ServerBuild/amazon-cloudwatch-agent.json in the Cloud9 editor by navigating to the file from the file browser on the left and double selecting it. Then, set the value of role_arn and save the file. Find the correct ARN in the output of the deployPreRequirements.sh script or in the CloudFormation stack GameLiftExamplePreRequirements Outputs.

AWS Cloud9 editor

Open the CppServerAndClient folder in the Cloud9 terminal and run the script to download, build, and setup the GameLift C++ Server SDK by executing the following commands:

cd CppServerAndClient

./SetupGameLiftSDK.sh.

This will take some time, so you can review the key parts of the script in the meantime. It will download the specified version of the GameLift Server SDK, build the SDK with cmake, and then copy the headers and binary files to the Server folder to be utilized by our server program.

Then, open CppServerAndClient/BuildAndDeployCppGameServerAndUpdateGameLiftResources.sh in Cloud9. Here you could set the values of region and secondaryregion to the appropriate values. Leave them to the defaults (“us-east-1” for region and “eu-west-1” for secondaryregion).

AWS Cloud9 editor

Now the server can be built, uploaded to GameLift, and the CloudFormation stack can be deployed to create every GameLift resource. This is done by running the following command in the Cloud9 terminal:

./BuildAndDeployCppGameServerAndUpdateGameLiftResources.sh

The process will take time, as it will build and setup every GameLift resource and deploy a fleet with game server instances in two different regions. You can follow the progress in the CloudFormation and GameLift management consoles. You should eventually see two locations as “Active” for the GameLift Fleet by navigating to the Fleet and selecting Locations in the GameLift management console.

Building and running game clients
Now you should have the backend, game server, and the Amazon GameLift resources deployed. You can now build and run two test clients to test the solution end to end. For this example, you will build and run the clients on Linux in Cloud9.

Let’s first review some of the key functionalities of the client in CppServerAndClient/Client/Client.cpp.

In the main() function, the example sets up the AWS SDK and creates an Amazon Cognito client for the backend resource region. It then requests an identity from the identity pool. Note that this example doesn’t cache the Cognito ID for future use, and it will request a new one when connecting. However, you can store this identity on the client and use it to request the credentials directly on subsequent launches. Also, the credentials expire in 1 hour, so you need to call GetCredentialsForIdentity hourly in order to maintain valid credentials. This example uses an unauthenticated identity, but you can link Cognito to external identity providers and even use custom Developer Authenticated Identities for any custom integrations, such as game publishing platform specific identities.

In the GetLatencyString() function, the client measures latencies through DynamoDB endpoints for GameLift region locations. DynamoDB is used because it’s available in every applicable region, and it replies with a “healthy” status to an HTTPS request. These latencies are the average of two HTTPS requests after establishing a connection, which gives you a good idea of the TCP latency. The first request is not used to eliminate the initial TCP and TLS handshake delays from the measurement.

Then, the client utilizes the Cognito provided credentials in order to sign requests against our API. The AWS SDK for C++ contains the AWSAuthV4Signer class to simplify the signing process.

Run the following script in the Cloud9 terminal to download, build, and configure the AWS SDK for C++:

./SetupAWSSDK.sh

This script will make the AWS Core and Amazon Cognito libraries available on this instance. This is so you don’t need to copy them separately. The script will download the AWS SDK for C++ and utilize cmake to build, and then make to install the relevant components of the SDK. You only need the module cognito-identity, so set the-DBUILD_ONLY flag only for that module. It will still automatically build every dependency of that module.

Then, open CppServerAndClient/Client/Client.h in Cloud9 and configure the correct endpoints for Cognito and API Gateway:

Set the value of String backendApiUrl to the endpoint created by the backend deployment. Find this endpoint from the gameservice-backend Stack outputs in CloudFormation, from the SAM CLI stack deployment outputs in the Cloud9 terminal, or from the Lambda Application view used earlier.
Set the value of String identityPoolId to the identity pool created by the pre-requirements deployment. Find the ARN in the CloudFormation stack outputs, in the Amazon Cognito management console or as the output of the deployment command in the Cloud9 terminal.
Leave the values of const char* REGION and String regionString to the default “us-east-1”, as you are using this across the deployment. Leave the value of String secondaryRegionString to “eu-west-1” for the secondary location of the GameLift Fleet. This configuration will define where the backend and fleets are located. It is also used to make backend API requests and measure latencies to the different regions on the client side.

AWS Cloud9 editor

Then, run two clients to test the end-to-end flow of the solution:

Navigate to the folder CppServerAndClient/Client/ in two Cloud9 terminals. You can create a new one from the “+” icon menu.
Run ./build.sh in one of the terminals to build the client.
Run ./client in both terminals to start two clients.
You should see the clients requesting a Cognito identity, measuring the latencies against the regional endpoints, and then requesting matchmaking and periodically checking the status of their matchmaking ticket. Finally, they will connect to the server and send their Player Session ID for validation and receive a result back.

AWS Cloud9 terminal

Conclusion
You should now have a working end-to-end solution for an online multiplayer game written in C++. The backend utilizes serverless services and scales automatically to your needs. Game servers are hosted in two different locations and you can add new regions easily as needed for a global presence. The game client will check its latency against these regions, and GameLift FlexMatch will utilize the latency information to group players together with others close to them.