Unity UNET HLAPI and Steam P2P networking

About

This article is intended for intermediate Unity developers looking to integrate Unity Networking (UNET) with Steamworks peer-to-peer networking.

TL;DR: Show me the code!

Action shot of the example project Action shot of the example project

 

UNET and Steam P2P

So you've already implemented your game's multiplayer features with UNET and - wait, wtf? It doesn't even support Steam P2P?

Fret not! They're working on it. And for now, there are ways to trick UNET into working with Steam P2P. This is how I did it for our game Rival Megagun.

 

What's the point?

Lower cost.

This method is good for multiplayer games that use matchmaking and P2P networking but don't require dedicated servers. By using UNET and Steam's NAT-traversal and relay servers, you don't need to pay for Unity's multiplayer services or host your own facilitator/relay servers.

 

Is this the best method?

I don't know. 

This implementation makes some assumptions about the internals of UNET, so it may require some tweaking if UNET has any major updates. To fully understand it, you may need to dig into UNET's source code. I like this method because I didn't have to modify the UNET source code, or install any extra third party plugins, or pay for any extra services. 

 

How does it work?

See for yourself.

Essentially we override UNET's transport code to use the Steam P2P API. This allows UNET to continue to function as expected on the surface while using Steam to send and receive data. All of your UNET RPCs, NetworkBehaviours, NetworkMessages, etc. will continue to work as intended. 

The general rule of thumb is that any function that uses UnityEngine.Networking.NetworkTransport needs to be replaced with a custom implementation. Connecting, disconnecting, sending and receiving data, all rely on NetworkTransport and need to be replaced.

This isn't a full tutorial. Just highlighting some examples. If you want to see the full working solution, check out my sample project. 

1. Connecting

1.a) Start UNET Server

Start the NetworkServer on the host machine and add a NetworkClient to represent the local client (just as you would with any UNET game). The NetworkServer should not actually listen on any port. This is because we don't transmit data through UNET, we do so through Steam P2P.

void StartUNETServer()
{
    // Start UNET server
    NetworkServer.Configure(SteamNetworkManager.hostTopology);
    NetworkServer.dontListen = true;
    NetworkServer.Listen(0);

    // Create a local client-to-server connection to the "server"
    // Connect to localhost to trick UNET's ConnectState state to "Connected", which allows data to pass through TransportSend
    myClient = ClientScene.ConnectLocalServer();
    myClient.Configure(SteamNetworkManager.hostTopology);
    myClient.Connect("localhost", 0);
    myClient.connection.ForceInitialize();

    // Add local client to our list of connections. Here we get the connection from the NetworkServer because it represents the server-to-client connection
    var serverToClientConn = NetworkServer.connections[0];
    connectedClients.Add(serverToClientConn);
}

1.b) Establish P2P Connection

Establish a P2P connection through Steam. This is done by first starting a Steam lobby on the host machine. Steam clients join the lobby and send a packet to the host to request a P2P connection. The host accepts this connection and sends a packet back as confirmation. 

A NetworkConnection is then instantiated on both machines to represent the P2P connection. You will need to create a class derived from NetworkConnection so that it can reference the peer's SteamID.  (e.g. SteamNetworkConnection)

Client:

// Called on Client by OnLobbyEntered
IEnumerator RequestP2PConnectionWithHost()
{
    var hostUserId = SteamMatchmaking.GetLobbyOwner (steamLobbyId);

    //send packet to request connection to host via Steam's NAT punch or relay servers
    SteamNetworking.SendP2PPacket (hostUserId, null, 0, EP2PSend.k_EP2PSendReliable);
  
   // wait for response from host
    uint packetSize;
    while (!SteamNetworking.IsP2PPacketAvailable (out packetSize)) {
        yield return null;
    }

    byte[] data = new byte[packetSize];
    CSteamID senderId;

    if (SteamNetworking.ReadP2PPacket (data, packetSize, out packetSize, out senderId)) 
    {
        if (senderId.m_SteamID == hostUserId.m_SteamID)
        {
            // packet was from host, assume it's notifying client that AcceptP2PSessionWithUser was called
            P2PSessionState_t sessionState;
            if (SteamNetworking.GetP2PSessionState (hostUserId, out sessionState)) 
            {
                // Connect to the unet server 
                // Create connection to host player's steam ID
                var conn = new SteamNetworkConnection(hostUserId);
                var mySteamClient = new SteamNetworkClient(conn);
                this.myClient = mySteamClient;

                // Setup and connect
                mySteamClient.SetNetworkConnectionClass<SteamNetworkConnection>();
                mySteamClient.Configure(SteamNetworkManager.hostTopology);
                mySteamClient.Connect();
            }

        }
    }

}

 

Host:

// Called on Host when a Client sends their first packet
void OnP2PSessionRequested(P2PSessionRequest_t pCallback)
{
    if (NetworkServer.active && SteamManager.Initialized) 
    {
        // Accept the connection if this user is in the lobby
        int numMembers = SteamMatchmaking.GetNumLobbyMembers(SteamLobbyID);

        for (int i = 0; i < numMembers; i++) 
        {
            var member = SteamMatchmaking.GetLobbyMemberByIndex (SteamLobbyID, i);

            if (member.m_SteamID == pCallback.m_steamIDRemote.m_SteamID)
            {
                // accept connection
                SteamNetworking.AcceptP2PSessionWithUser (pCallback.m_steamIDRemote);

                // send confirmation packet to peer
                SteamNetworking.SendP2PPacket (pCallback.m_steamIDRemote, null, 0, EP2PSend.k_EP2PSendReliable);

                // create new connnection for this client and connect them to server
                var newConn = new SteamNetworkConnection(member);
                newConn.ForceInitialize();

                NetworkServer.AddExternalConnection(newConn);
                connectedClients.Add(conn);

                return;
            }
        }
    }

}   

1.c) Initialization 

When starting the NetworkServer and instantiating NetworkClients and NetworkConnections, make sure you connect and initialize them properly. See my examples:

SteamNetworkClient.Connect()

UNETExtensions.ForceInitialize(NetworkConnection) 

UNETServerController.StartUNETServer ()

2. Sending data

Override NetworkConnection's TransportSend function to send data directly to the peer via the Steam P2P API. 

public class SteamNetworkConnection : NetworkConnection
{
    public CSteamID steamId;

    public SteamNetworkConnection() : base()
    {
    }

    public SteamNetworkConnection(CSteamID steamId)
    {
        this.steamId = steamId;
    }

    public override bool TransportSend(byte[] bytes, int numBytes, int channelId, out byte error)
    {
        if (steamId.m_SteamID == SteamUser.GetSteamID().m_SteamID)
        {
            // sending to self. short circuit
            TransportReceive(bytes, numBytes, channelId);
            error = 0;
            return true;
        }

        EP2PSend eP2PSendType = EP2PSend.k_EP2PSendReliable;

        QosType qos = SteamNetworkManager.hostTopology.DefaultConfig.Channels[channelId].QOS;
        if (qos == QosType.Unreliable || qos == QosType.UnreliableFragmented || qos == QosType.UnreliableSequenced)
        {
            eP2PSendType = EP2PSend.k_EP2PSendUnreliable;
        }

        // Send packet to peer through Steam
        if (SteamNetworking.SendP2PPacket(steamId, bytes, (uint)numBytes, eP2PSendType))
        {
            error = 0;
            return true;
        }
        else
        {
            error = 1;
            return false;
        }
    }

}

3. Receiving data

Poll for P2P packets and pass the data to the appropriate NetworkConnection's TransportReceieve function. UNET will handle the rest. 

void Update()
{
    if (!SteamManager.Initialized)
    {
        return;
    }

    if (!IsConnectedToUNETServer())
    {
        return;
    }

    uint packetSize;

    // Read Steam packets
    while (SteamNetworking.IsP2PPacketAvailable (out packetSize))
    {
        byte[] data = new byte[packetSize];

        CSteamID senderId;

        if (SteamNetworking.ReadP2PPacket (data, packetSize, out packetSize, out senderId)) 
        {
            NetworkConnection conn;

            if (UNETServerController.IsHostingServer())
            {
                // We are the server, one of our clients will handle this packet
                conn = UNETServerController.GetClient(senderId);
            }
            else
            {
                // We are a client, we only have one connection (the server).
                conn = myClient.connection;
            }

            if (conn != null)
            {
                // Handle Steam packet through UNET
                conn.TransportReceive(data, Convert.ToInt32(packetSize), 0);
            }

        }
    }

}

 

Where can I learn more?

If you're looking for some more information on this topic, I found these posts very helpful: here and here.

Learn more about UNET, Steamworks, and Steamworks.NET.

Familiarize yourself with the UNET source code.

Check out my sample project.

 

About the author

Justin Rempel is an independent game developer currently working on Rival Megagun, the PVP battle shmup. Check it out at rivalmegagun.com and on Twitter

61 Comments

Justin Rempel

Justin Rempel

I’m not sure what the best way is. I don’t know if it’s possible with NetworkTransport.GetCurrentRTT (I never got it to work).

What I would do is implement my own ping system using raw Steam packets.

  1. In a Coroutine, send a special “ping” packet to the server or clients every x seconds.
  2. When polling for packets in the SteamNetworkManager’s update loop, handle these special packets instead of passing them through UNET.
  3. If packet is a ping, send pong. If packet is a pong, update the connection’s last RTT (Round trip time).

Some (very hacky) code to demonstrate this: Github

Joan6694

Joan6694

Hi, First I’d like to say thank you for this, you saved me a lot of troubles here. I started integrating your solution to my game but some of the code is relying on the method “OnStartLocalPlayer” from the NetworkBehaviour class (which is not called for some reason). I’m not sure if this happens only on my side or if it’s global but I wanted to report this issue in any case. I’m in the process of fixing this myself, but if you have any idea about what could be happening, I’m all ears. Thank you again!

Justin Rempel

Justin Rempel

Thanks for the kind words. Glad you have found it useful!

My example project does not use UNET’s player object system. I haven’t tested this, but you might need to register the player objects before spawning them. Try this - in UnetServerController.cs, add the following lines to the SpawnPlayer function:

ClientScene.AddPlayer(conn, 0);
NetworkServer.AddPlayerForConnection(conn, player, 0);

Hope this helps!

Joan6694

Joan6694

Hello again! Sorry for the late response, didn’t see you actually answered me. In the end I managed to solve my problem by replacing NetworkServer.SpawnWithClientAuthority(player, conn) by NetworkServer.Spawn(player); NetworkServer.AddPlayerForConnection(conn, player, 0);

Since it’s not the same thing as your solution, I just hope it won’t break on the long run. But anyway, that not why I’m here.

I managed to reproduce the behaviour of NetworkLobbyManager with your code, and it works well, but I’m hitting another wall now. I’d like to get the delay between my players to implement some kind of lag compensating method, but unfortunately NetworkTransport.GetRemoteDelayTimeMS gives me errors (mainly “WrongConnection” and “host id out of bound”). I’m guessing those errors come from the fact that NetworkTransport is missing something internally, but I can’t check since the sources of NetworkTransport don’t seem to be available. Of course you don’t have access to the sources either, but if you have any hint about how to fix that, it would be amazing.

In any case, thank you again for your work!

Justin Rempel

Justin Rempel

Yeah, that’s the idea – generally anything that hits NetworkTransport needs a custom implementation. As far as I know you have to roll your own solution. I have a rough ping example here if it helps.

Joan6694

Joan6694

I see, it’s a shame really. Anyway, thank you for the code, it will definitely help. I’ll get back to you if I have more questions.

Have a nice day!

Justin Rempel

Justin Rempel

Yes, this is using Steam P2P.

You can invite your friends by calling SteamFriends.ActivateGameOverlayInviteDialog(). This either triggers the GameLobbyJoinRequested_t callback or sets the command line arguments on the invitee’s game. These can be used to retrieve the lobby ID.

Following that, the invitee can connect to the server as described in “How Does It Work?” Section 1.B.

For more information check out the Steamworks docs here:

Steam docs

Here are some starting points in my example project:

Github 1

Github 2

Github 3

Tom

Tom

Hi again, Been trying to put this into my project but I’m having issues, I have currently built my unet and used you’re unet steam version and for some reason objects which are networked (i.e spawned on start) and aren’t related to the player, just aren’t spawning i.e staying disabled when they are meant to be enabled. On my Unet it works fine and everything enables but I just can’t work it out for the life of me

Any advice?

Thanks

Justin Rempel

Justin Rempel

If you want to detect when the host leaves, you can use SteamMatchmaking.GetLobbyOwner and compare it to pCallback.m_ulSteamIDUserChanged in the LobbyChatUpdate callback.

If the local player is the server and you need to detect when they disconnect it’s a bit trickier. You will have to manually invoke MsgType.Disconnect. There are a few things you can do, like poll the network status in a coroutine, set up a ping system, or roll your own event system and hook it into the SteamNetworkManager or UNETServerController Disconnect function.

Justin Rempel

Justin Rempel

Update - I may be wrong about using LobbyChatUpdate to detect when the host leaves. You could try using LobbyDataUpdate instead. Rough example here

Justin Rempel

Justin Rempel

Hi,

You should be able to switch scenes using Unity’s Network Manager. Just modify the SteamNetworkManager so that it extends Unity’s NetworkManager. If you’re not using Unity’s Network Manager, you’ll have to implement your own scene switch system.

https://docs.unity3d.com/ScriptReference/Networking.NetworkManager.ServerChangeScene.html

Aron

Aron

Hi again, I’m getting this error “Unknown message ID 36 connId:0” Which I believe is because we have registered the packet. How do I fix this?

Justin Rempel

Justin Rempel

Message 36 is the “NotReady” message. You can register using NetworkServer.RegisterHandler and NetworkClient.RegisterHandler.

Message IDs can be found here.

chad franklin

chad franklin

Hello, I am trying to send messages from client to server and also from server to client. You show us an example of client to server when connecting: myClient.Send(NetworkMessages.SpawnRequestMsg, new StringMessage(SteamUser.GetSteamID().m_SteamID.ToString()));

but, we don’t have an example of the server sending to clients. When I try to use the send method on the local client I get this error:

NetworkClient Send when not connected to a server UnityEngine.Networking.NetworkClient:Send(Int16, MessageBase)

Do you know how I can fix this?

Also, how are you using the Steam P2P sending and receiving data? We can’t find any examples in the project.

Justin Rempel

Justin Rempel

Hi,

You could keep a reference of the lobby owner’s steam ID and check if it changed in the LobyDataUpdate callback.

Rough example of this here.

chad franklin

chad franklin

How do I correctly disconnect the host? When I disconnect the host using the Disconnect method in SteamNetworkManager, while a client is connected, I get this error on the host:

host id out of bound id {1} max id should be greater 0 and less than {1} UnityEngine.Networking.NetworkServer:Shutdown()

It doesn’t seem to break the game, but when I repeat the process (without restarting the game) the error changes:

host id out of bound id {3} max id should be greater 0 and less than {1} UnityEngine.Networking.NetworkServer:Shutdown()

Should I be removing clients or something before I disconnect the host? Thanks for getting me this far.

Justin Rempel

Justin Rempel

Not sure… I will have to dig around to find the issue. In the meantime you could try this:

In UNETExtensions.cs - UNETExtensions.ForceInitialize:

Change:

conn.Initialize(“localhost”, id, id, SteamNetworkManager.hostTopology);

To:

conn.Initialize(“localhost”, 0, id, SteamNetworkManager.hostTopology);

chad franklin

chad franklin

Alright, I will try that. If it helps, I have run into another problem. When I disconnect the client from the host, it’s connection remains in NetworkServer.connections and it is NOT null. Idk how this is possible.

Justin Rempel

Justin Rempel

No I haven’t tried it. You might have to do something custom for that. The general rule of thumb is that anything that hits NetworkTransport needs a custom implementation.

chad franklin

chad franklin

Sorry for all of the questions, but i ran into another problem. I don’t think I’m disconnecting clients from the server properly. Do I just run the same Disconnect function in SteamNetworkManager? I have set up a method to disconnect the client from the server if the server is already in the game scene (opposed to the lobby scene) and all it does is send a message to the client to make it call the disconnect function. But after doing this, I am getting this log about every 15 seconds on the Client:

P2P session request received

chad franklin

chad franklin

Alright, turns out this one was some foolish user error om my part. Sorry if I wasted any of your time.

Justin Rempel

Justin Rempel

Not sure. The Steam API might have support for it. Or you could do something like send a NetworkMessage and call CloseP2PSession.

chad franklin

chad franklin

Hello again, do you have the problem of the client crashing if the host does Alt-f4? How would I prevent the client from crashing?

Justin Rempel

Justin Rempel

I haven’t experienced this issue so I can only guess… Maybe the client is stuck in an infinite loop of calling Disconnect.

Ziggy Shaw

Ziggy Shaw

I cant get the public methods from the NetworkManager to call each other. For example if I add a new player with: ClientScene.AddPlayer(conn, 0); (within the UnetServerController.cs for example) It should call NetworkManager.OnServerAddPlayer, eg:

public override void OnServerAddPlayer(NetworkConnection conn, short playerControllerId) { Debug.Log(“OnServerAddPlayer”); }

but it doesn’t. This is the same with all the other “OnClient*” methods. Is this a limitation with using a custom server controller or is there something I’m missing?

It’d be infinitely useful if there was a way to retain these methods.

Thank you so much anyway. This is seriously amazing.

Justin Rempel

Justin Rempel

To keep this example simple I did not use the Unity Network Manager but you can easily add it. Does your custom network manager extend the Unity network manager?

e.g.

public class SteamNetworkManager : NetworkManager { }

Ziggy Shaw

Ziggy Shaw

Yes it does. To test it, on your github example I simply changed the network manager to public class SteamNetworkManager : NetworkManager {

and added the function to the SteamNetworkManager class:

public override void OnServerAddPlayer(NetworkConnection conn, short playerControllerId) { Debug.Log(“OnServerAddPlayer”); }

I then added ClientScene.AddPlayer(conn, 0); to the SpawnPlayer method.

The debug log “OnServerAddPlayer” doesnt present itself, even though the SpawnPlayer method completes.

I have the same problem with OnClientSceneChanged. It seems to be a problem with overriden NetworkManager methods.

Ziggy Shaw

Ziggy Shaw

https://answers.unity.com/answers/1442685/view.html

This guy seems to suggest its to do with RegisterHandler replacing other handlers. Do you know what I should change to allow for these functions to be called?

Justin Rempel

Justin Rempel

Don’t know off the top. I’ll have to look into it.

Not sure if it helps but I have solved similar issues by calling these functions. Might be worth looking into:

NetworkServer.Spawn(player); NetworkServer.AddPlayerForConnection(conn, player, 0);

Ziggy Shaw

Ziggy Shaw

Yeah I saw that that comment.

The NetworkServer.AddPlayerForConnection(conn, player, 0); method calls the NetworkBehaviour.OnStartLocalPlayer method. Which works completely fine.

My problem is that NetworkManager On* methods arent being called.

Ziggy Shaw

Ziggy Shaw

Ok I think I understand the problem. I’m just not sure how to solve it.

I think due to the fact that we’re not running the server via NetworkManager.StartServer() https://github.com/jameslinden/unity-decompiled/blob/master/UnityEngine.Networking/NetworkManager.cs#L722

We never run this.RegisterServerMessages(); https://github.com/jameslinden/unity-decompiled/blob/master/UnityEngine.Networking/NetworkManager.cs#L770

Which basically connects the rest of the functionality of the NetworkManager up.

The problem is that RegisterServerMessages along with a lot of other useful classes are all internal so they’re only visable to the same assembly body, so you cant access them with inheretence.

Justin Rempel

Justin Rempel

Nice find. Okay, this is fixable by swapping out NetworkServer.Listen() with SteamNetworkManager.Instance.StartServer(). This will register the message handlers.

When calling ClientScene.AddPlayer for the host’s player, make sure you use the client-to-server connection (the one created with ClientScene.ConnectLocalServer).

Hope that helps

Jake

Jake

Hello Justin, First thank you for posting this - it’s a big help. I’m currently studying your code and also the code here: http://www.unitystudygroup.com/unetsteam

Is this guy using the same technique you are using? If not, could you tell me what the primary differences are? I know I’m asking you to do my homework, but I’m pretty new to UNET and I’m just hoping for some pointers. His code seems much shorter, so I’m inclined to think it’s missing some functionality that yours has, just not sure what. Anyway just thought I’d ask. Thanks!

Jake

Justin Rempel

Justin Rempel

Hi Jake,

It looks like they are using game servers. I am using peer-to-peer networking, which is a bit different.

Steam’s peer-to-peer networking supports NAT punch and relay servers, which means your players don’t need to configure port forwarding on their routers. I don’t know if the Unity Study Group’s solution utilizes that. It’s worth looking into.

I would recommend experimenting with a few different solutions to figure out what works best with your project. Also, do some research on the Steamworks docs if you haven’t yet.

https://partner.steamgames.com/doc/features/multiplayer

Justin

Jake

Jake

Hi Justin,

Yep I just spent an hour or two digging through his stuff and kinda figured that out too. He’s just using the steam match data to share his IP with other clients. They then connect normally via UNET, and thus don’t get any benefit of P2P or NAT punch. I do appreciate you responding to me so quickly! And thanks for the info!

Jake

Jman

Jman

I’ve got a really weird bug after switching to this system but it’s not awful, it is technically gamebreaking but I can work around it.

Basically all SyncVars run multiple times, which means SyncVar hooks run multiple times, I was using these for toggling an ability on/off and it would add/remove some attack speed depending on the ability state. After switching to this it was removing the attack speed 5x causing the player to get slower and slower as they used it.

The call stack showed the SyncVar hook being run locally as I was hosting but then it showed it changing again from UNetStaticUpdate and running multiple times.

Anyways I just moved all my hooks to Rpc and it’s fine now but yeah none of them worked after swapping to SteamP2P for whatever reason.

Justin Rempel

Justin Rempel

You can modify the SteamNetworkManager to extend Unity’s Network Manager to use its features if you wish.

Daniel

Daniel

Hi again!, I already have all the connection and scenes change working. I have only one question, I have a server browser that display the host user name and current players in the lobby, how can I show the ping I have with the lobby in the server browser?. Thanks you very much.

Justin Rempel

Justin Rempel

If you’re using GameServers, this might be a good starting point.

I don’t know if that will work if you’re using a pure P2P solution. You might have to implement something custom. I wrote a rough example for retrieving ping from a peer here:

Comment

Code

Jose

Jose

Hello and thanks for this method to implement steam networking… I have a few questions, -How can I kick a player from the server and make that player exit to the menu?, -How can I check for connection timeout?

Martin

Martin

Hi, im trying to understand ur code but still it’s difficult for me. I want to change scene when all guys are in lobby but i don’t know how.. Can you give me a clue or maybe code? Btw. i saw ur post with “https://docs.unity3d.com/ScriptReference/Networking.NetworkManager.ServerChangeScene.html “ but that’s so hard for me..

Leave a Comment

Sending...