Revisiting Fixing Valheim Lag – Modifying Send/Receive Limits

My Valheim Dedicated Server Castle
My Valheim Dedicated Server Castle

I’ve previously covered fixing Valheim’s dedicated server lag by modifying the dedicated server’s send receive limits. Since I wrote that article not long after launch though the game has been updated that the old method no longer applies. I had theorized in the comments of that article how to address this and today Sam R. confirmed that this method did indeed alleviate most congestion on the server in the places that the send/receive limits are known to bottleneck.

This updated guide will show some updated methods you can use to help reduce lag on your Valheim dedicated server!

Symptoms of lag on Valheim dedicated servers

The symptoms you are experiencing on the server can help narrow down what type of lag you are experiencing and what may be causing it. Lag caused by your CPU being too overloaded for example will have different symptoms than lag that is caused by your server hitting the maximum send/receive limit as an example.

By far the most common type of dedicated server lag seen in the early access Valheim release so far has been mainly due to the low send/receive limit if 64KB/s that is hard coded into the server. This means that when you do something like take your character a heavily populated area on the server it has to send all that information about all the buildings/players through an artificially limited very narrow 64KB/s pipe.

Symptoms of the send/receive limit type of lag include things like chests taking a very long time to open, sometimes 10-30 seconds or never opening while at the same time being able to see players walking around normally. This is because the other people’s X / Y / Z positions is a very small amount of data compared to all the items/names/attributes that are stored in the chest. The other players moving easily fits through the narrow pipe while the chest full of loot’s data chokes since it is much larger.

I’ll specifically lay out how to modify your server and increase this limit in the “Modifying your server’s send/receive limit” section below. Before that though let’s cover a more specific type of lag to watch out for (mostly by being avoided) related to terrain manipulation.

Avoid extensive terrain manipulation (for now)

Another commonly observed source of lag has to do with terrain manipulation. Some players have found that if they take a big chunk out of the ground in Valheim and make extensive terrain modifications that the area gets filled with separate “instances” of terrain for each modification people make. Think of the original ground as one big piece in the game files. Valheim has the very cool feature of letting you modify terrain by raising it/lower it/level it/etc. Each time you do this though the game has to store what you did as a new “instance” or a modification.

The way Valheim seems to work now is it renders that original piece and then the modifications afterward. Early reports suggest having a whole bunch of them in one spot seems to sometimes cause everyone’s in-game FPS on the server to drop as low as 40 when the clients normally are much much higher than that.

The best way to avoid this type of lag at the moment is to limit how much terrain modification you are doing in areas near your base. It is likely this will be addressed/eased in future patches. There should be some things the developer team can do to clean up/merge these instances or at least reduce their effect on the rendering process so it doesn’t drop the in-game FPS so dramatically.

Prebuilt Binaries Links (Added 1/10/2023)

Sketaful has volunteered to put together binaries of the modification in this article.

His file repository is located here.

Thank you for doing this Sketaful for as long as you are able. If the files are no longer working let me know in the comments and I’ll remove the link but as long as Sketaful wants to compile the binaries I’ll link to them here!

If you want to use Sketaful’s binaries you can skip way down to the “Backing up original file and replacing with our new one” section. That will tell you where this file needs to go and let you safely back up your original one first.

Modifying your server’s send/receive limit

At this time there is no options file for Valheim unfortunately. The only way to change it is to modify the server’s code and recompile the module that controls this.

Fortunately this is a lot easier than it sounds because the module we need to change is in .NET instead of something like C++ which would be much more difficult to do this with. I did not come up with this method, a very clever user on reddit (maximgame) shared it here and I want to give credit where credit is due!

Despite it not being too difficult these modifications are at your own risk. I haven’t experienced any problems so far but technically we are modifying the server’s code with the following procedure so understand that this isn’t an officially supported method and could have side effects or if you make a mistake could cause serious problems. Make sure you have backed up everything (your game world files especially) before you try anything!

Another important note is that it’s very likely you will have to make the changes again each update since the file we are going to modify is going to get replaced by updates. Until they add either a launch option or options file to officially change/control this property that is unfortunately unavoidable.

With all that being said if you understand the above and still feel comfortable let’s proceed!

Get dnSpy utility

We are going to use the utility dnSpy to modify the server code and recompile the module. The utility is free and is available here (download the Win64 build or dnSpy-net-win64.zip): GitHub dnSpy Official Page

Extract the archive and run the program dnSpy.exe from inside the folder you extracted. The dnSpy application will open.

Modifying server with dnSpy

We are going to go to the “File” menu in the top left and choose “Open” like this:

dnSpy - File -> Open Menu
dnSpy – File -> Open Menu

Now you need to navigate to the Valheim dedicated server folder which is located in your “steamapps” folder. If you left all the options default when you installed it’s typically “C:\Program Files (x86)\Steam\steamapps\common\Valheim dedicated server\valheim_server_Data\Managed”.

We are looking for the file “assembly_valheim.dll” in that folder:

dnSpy - Select File
dnSpy – Selecting Valheim server assembly file

Select the file “assembly_valheim.dll” and click “Open” and the server’s assembly will load into dnSpy.

Once it has finished loading we are going to go to the “Edit” menu and click “Search Assemblies” like this:

dnSpy - Search Assemblies
dnSpy – Search Assemblies

For the search criteria enter “ZDOMan” like this:

dnSpy - ZDOMan Search
dnSpy – Searching for ‘ZDOMan’

After the search completes you should see a list of results like the one above. Double click on one of these entries on the list and it should place you in the ZDOMan section of the assembly (seen on the left hand side of the screen):

dnSpy - 'ZDOMan' section of assembly
dnSpy – ‘ZDOMan’ section of assembly

Perfect. We are now in the right place to make the changes. We just need to locate the line that has the send/receive limitation. To find that press Ctrl+F to bring up the find box (or choose “Edit” -> Find from the menu) and search for:

 sendQueueSize

which should bring up the following lines:

dnSpy - sendQueueSize Variable
dnSpy – sendQueueSize Variable

If you are having trouble finding the lines make sure you’re using Ctrl+F to search this time and not the “Search Assemblies” from earlier at the bottom of the screenshot, that’s a different type of search. Make sure you’ve disabled the case sensitive options (the Aa right below the box) or any other search filters. Your search query box should be in the top right of the screen like the above screenshot that has sendQueueSize in it.

Now right click on this line:

int num = 10240 - sendQueueSize

and click “Edit Class (C#)…”. Another window will open that will let you make changes to the file. We are going to modify ‘10240’ to a variable of our choosing. I made mine 30720 since after doing some conversion math that is the equivalent increase from my previous launch version guide that provided great performance and I kept them the same. If you followed my last guide this will give you the exact same performance as the old one did.

There are two instances of this variable. The second instance is 3 lines above the highlighted line we searched for and is:

if (!flush && sendQueueSize > 10240)

Modify both instances of 10240 to 30720.

Theoretically setting a limit too high could cause the server to swamp your internet connection if you have a large number of players on and there’s essentially no limit but there shouldn’t be any other effects of a very high number.

Once you’ve changed both numbers to the desired value we need to press the “Compile” button in the bottom right corner of this new window. This window will close and take you back to the previous one.

Saving the new assembly

It’s time to save our changes into a new .dll file. Choose “File” -> “Save Module…”:

Saving New Assembly
dnSpy – Save Module as assembly_valheim_modded.dll

Change the file name to assembly_valheim_modded.dll. Don’t overwrite the original file just yet as we will want to take a backup and make sure that the server is closed before we do that. Press “OK” to write the new assembly_valheim_modded.dll file.

Backing up original file and replacing with our new one

Now it’s time to back up the original file and replace it with our new modded file. First make sure your dedicated server is closed otherwise the dll file will be in use. It is also a good time to make a backup of your world file just in case (see my Valheim Dedicated Server Backup Location / Guide for instructions)

Navigate in a file explorer window to your Valheim server directory where we saved the new assembly file (usually C:\Program Files (x86)\Steam\steamapps\common\Valheim dedicated server\valheim_server_Data\Managed).

First make a copy of the original assembly_valheim.dll file. You can just select the file and “copy” then paste a copy of it in that same folder and Windows will make a “valheim_assembly Copy(1).dll” file for you or you can back it up to a separate folder. Now remove the original file and rename your “valheim_assembly_modded.dll” file to the original file name of “valheim_assembly.dll”.

That’s it, you’ve now replaced the old assembly with the new one! Go ahead and start your server normally and the changes will have taken effect.

Other Resources

Make sure you are taking constant backups of your Valheim dedicated server: Backup Valheim Dedicated Server – World Folder Paths / Guide

It’s also important to back up your local game’s characters and local worlds if you are using them. I have written a guide for the local game side that is available here!

For historical reasons I’ve left up the post with the old method here as it helped me devise the new method: Fixing Valheim’s dedicated server lag by modifying the dedicated server’s send receive limits

Other Valheim Lag Issues

If you know of other Valheim lag issues that I didn’t cover here or are still having problems let me know in the comments and I will update this guide with other tips/tricks and try to answer any questions. It’s still a very new game and there isn’t very much out there for it yet. Have fun and enjoy!

guest

190 Comments
Inline Feedbacks
View all comments

4LL
4LL
4 days ago

Dropping a comment to say that following both yours and Patk88’s suggestions, I’m hosting a server for several people running epicloot and valheim plus – no major lag!

I’ve consolidated the instructions below, this takes less than 5 minutes:

************
**SERVER**
************

1. Use dnSpy to open:
C:\Program Files (x86)\Steam\steamapps\common\Valheim dedicated server\valheim_server_Data\Managed\

[[ James A Chamber sendQueueSize Fix ]]

2. Within dnSpy toolbar { Edit -> Search Assemblies } or Windows Hotkey { Ctrl+Shift+K } to search for:
ZDOMan

3. Within ZDOMan file use dnSpy toolbar { Edit -> Find} or Windows Hotkey { Ctrl+F } to search for:
sendQueueSize

4. Right click on “int num = 10240 – sendQueueSize” and select Edit Class (C#)

5. Replace both 10240 to 30720

6. Compile

[[ patk88’s Server Fix ]]

7. Search Assemblies { Ctrl+Shift+K } to search for:
zSteamSocket

8. Within zSteamSocket search { Ctrl+F } for:
RegisterGlobalCallbacks
and edit its class

9. Find gchandle3 and replace its number with 524288

10. Insert the following lines of text just before gchandle.Free()

SteamGameServerNetworkingUtils.SetConfigValue(ESteamNetworkingConfigValue.k_ESteamNetworkingConfig_SendBufferSize, ESteamNetworkingConfigScope.k_ESteamNetworkingConfig_Global, IntPtr.Zero, ESteamNetworkingConfigDataType.k_ESteamNetworkingConfig_Int32, gchandle3.AddrOfPinnedObject());

11. Compile

[[ SAVE ]]

12. Choose “File” -> “Save Module…”
Change the file name to assembly_valheim_modded.dll

13. Choose “File” -> “Close All”

************
**CLIENT**
************

1. Use dnSpy to open:
C:\Program Files (x86)\Steam\steamapps\common\Valheim\valheim_Data\Managed

2. Search Assemblies { Ctrl+Shift+K } to search for:
zSteamSocket

3. Within zSteamSocket search { Ctrl+F } for:
RegisterGlobalCallbacks
and edit its class

4. Find gchandle3 and replace its number with 524288

5. Insert the following lines of text just before gchandle.Free()

SteamNetworkingUtils.SetConfigValue(ESteamNetworkingConfigValue.k_ESteamNetworkingConfig_SendBufferSize, ESteamNetworkingConfigScope.k_ESteamNetworkingConfig_Global, IntPtr.Zero, ESteamNetworkingConfigDataType.k_ESteamNetworkingConfig_Int32, gchandle3.AddrOfPinnedObject());

6. Compile

[[ SAVE ]]

7. Choose “File” -> “Save Module…”
Change the file name to assembly_valheim_modded.dll

—————-

Alright great, now all you have to do is replace your servers assembly_valheim.dll with your new assembly_valheim_modded.dll!

And your client’s (and all connecting clients) .dll’s with your brand spankin new assembly_valheim_modded.dll!

jic
jic
4 days ago

Is this DLL change the same for Linux docker based hosted servers? I see the DLL in my server files, but anytime I try and change it with a modded compiled DLL, the server hangs on startup and never fully loads ….client is working fine

vi0lation
vi0lation
5 days ago

Hi James, starting a new post for this.

No need to apologize, I completely understand your frustrations, I too am (almost) at my wits end trying to make the issues discussed better, because as you said without a total rework of the multiplayer code, there is no solution – just things to mitigate the symptoms. I’ll post below some of the scenarios we have tried in our testing.

We run a dedicated linux server, using Gportal as the host.
We play on a heavily modded version of valheim (90+ mods)
On our Friday game nights, have 4-6 players online simultaneously, all usually fighting and playing together in the same zone/instance, and this is where the lag, rubberbanding, teleporting, chest/door delay, weird boar behaviour etc. etc. pop-up.

The 3 mods the supposedly help with network performance I have used, and am currently experimenting with are:

Network
Compress
BetterNetworking Valheim

Network: Have been using this pretty much from the get-go (2-3 months of running a server, and trying to find ways to make it less laggy) This is the mod that unlocks the send/rate and has so far shown to improve results, significantly (but the issues still happen with 4-5+ players, and depending on the circumstances of the game etc.)

Compress: Just installed this one, to see if it makes any difference, and waiting for a few game nights to observe results, currently its installed in tandem with the Network mod, which may or may not conflict with each other, if that is the case, will retest with just Compress mod installed.

BetterNetworking Valheim currently has an issue where it doesn’t work on Linux Dedicated Servers, the dev is aware and is working on a resolution – personally I have not tested this mod yet, but will once the fix is in place.

I have not tried direct assembly editing yet, and will include that into the testing with other variables in the coming weeks.

This past Friday, instead of our regular game night we did a testing night, loaded into the server with devconsole so i could spawn mobs and raids under controlled conditions and we can record results. The following results came from that night:

We tested several combinations of different things, including:

– Running the server using PlayFab/Crossplay Network Protocol (Vanilla)
– Running the server using PlayFab/Crossplay Network Protocol (with Network Mod)
– Running the server using Steamworks Network Protocol (Vanilla)
– Running the server using Steamworks Network Protocol (with Network Mod)
– Removal of JewelCrafting Mod
– Spawning fights and raids around the town, and in an empty area, etc.

The best combo so far is steamworks network protocol (crossplay disabled) with network mod, the removal of JC had no impact. After a bunch of testing the following is pretty much the ‘result’:

– Town will always be laggier, due to all the ‘stuff’
– Fights outside of town are much less laggy
– Using Playfab/Crossplay network protocol was massively laggy compared to steamworks, to the point where it was game breaking / unplayable (huge load times, doors taken 5+ seconds to open, etc) (rolling back to steamworks but keeping everything else the same showed a significant improvement, only lag was fighting mobs in town + some other minor hiccups.

For example with 4 of us we were able to fight 20 skeletons, or 15 draugrs, etc. with no lag witnessed for anyone. (In an empty zone, no player built anything, etc) this is all still highly connected to who ‘owns’ the instance of the zone this is happening in. we tested this as well by having everyone except the player with the lowest end machine stay connected, so they assume control of the zone, and then have people reconnect, even in this state there was no/very little lag fighting a significant quantity of mobs, that said – fighting the same amount in our town, that has a large amount of buildings, chests, crops, cooking stations, etc. the same test yielded significant lag when fighting mobs.

I will continue to test different scenarios and report back

vi0lation
vi0lation
5 days ago

All seems to add up indeed, I’m wondering with the introduction of Crossplay this is when several people have started reporting worse performance for Mistlands, as you now have to manually disable crossplay on a host to force it back to steamworks. And the difference between Crossplay/Playfab vs. Steamworks was night and day.

In terms of my understanding of zones (This all comes from hanging around several discords with fairly prominent valheim mod devs and gurus, so I am just repeating what seems to be the general consensus on zone information)

– Zones are 64×64 sized grids
– The first player to reach/load a zone becomes the owner
– That said, to further complicate things, its not exclusive control, another player that interacts with an object (for the first time) inside a zone even if they werent the first there, becomes the owner of that specific object inside the zone. When your watching the log output of bepinx when playing multiplayer, you will often see the console spam with ‘attempting to interact with xxxx – but i am not the owner’ messages, this is essentially your client trying to interact with say a chest, or door that someone interacted with earlier and first, and therefore control it, but due to the backlog of multiplayer lag, your client cannot receive a response until that backlog queue is cleared and the owner can reply with the return, hence the console spam of that message until network congestion clears.

There is a mod that visualizes zones inside the game engine: (It’s pretty interesting to see live.)

I’m currently looking at the config of the betternetworking_valheim mod, and there are several configurable options:

# Setting type: Options_NetworkQueueSize
# Default value: _1024
# Acceptable values: _10240, _1024, _512, _128, _10
Queue Size = _1024

I’m wondering if this is patching:

int num = 10240 - sendQueueSize
and
if (!flush && sendQueueSize > 10240)

In the ZDOMan Assembly inside assembly_valheim.dll as per your article.

And:

# Setting type: Options_NetworkSendRateMin
# Default value: _300
# Acceptable values: _400, _300, _200, _150, _100, _50
Minimum Send Rate = _300

# Setting type: Options_NetworkSendRateMax
# Default value: _INF
# Acceptable values: _INF, _400, _300, _200, _150, _100, _50
Maximum Send Rate = _INF

Which I’m wondering if its patching:

GCHandle gchandle3 = GCHandle.Alloc(153600, GCHandleType.Pinned);

In the zSteamSocket Assembly inside assembly_valheim.dll as per the method presented by Pat in this forum. I’ve asked the mod author to see if that is indeed what it is doing or not, would be interesting to know.

vi0lation
vi0lation
5 days ago

Would it be possible to list what version of Valheim the hosted modified assembly is for? Current version of valheim is 0.212.9, is the hosted assembly for this version? Thanks for all the research and hardwork investigating this!

vi0lation
vi0lation
5 days ago

thanks for the reply, all good – just modified the dll myself based on your instructions, and read the entirety of this thread. 100% the same boat me and the 4-5 ppl I play with are in. same lag, teleporting, rubberbanding, etc. etc when we are all in one instance. I have tried running client/server with smoothbrain-network mod (which supposedly unlocks the caps without having to edit the assembly) and just recently trying the comfy-compress mod, which supposedly compresses zdodata before transfer to cut down on amount, but no testing done on this yet if it makes it worse or better.

Will try the modded assembly on server, and clients and retest as well to see if i can add to the pool of tested scenarios.

CW_Jesse (Better Networking mod dev)
CW_Jesse (Better Networking mod dev)
4 days ago

Worth noting that crossplay does enable compression in vanilla Valheim. It just uses very slow compression, though it achieves a good compression ratio.

CW_Jesse (Better Networking mod dev)
CW_Jesse (Better Networking mod dev)
4 days ago

Furthermore, ZPlayFabSocket’s.GetSendQueueSize() multiplies its result by 0.25, presumably to send more through the compressed socket. Dividing the result like that instead of subtracting 30KB (for a total effective queue size of 40KB) means slightly inconsistent amounts of data are sent.

CW_Jesse (Better Networking mod dev)
CW_Jesse (Better Networking mod dev)
4 days ago

That sounds like a reasonable conclusion for people to have reached given how they reached it, but also a wrong one given both my understanding of how Valheim’s networking works and my in-game testing. Better Networking allows you to change the queue size on the fly and that’s never caused an issue. It’s even beneficial to have people on different queue sizes based on their available upload bandwidth. Since you’re modifying the assembly directly, I’d sooner reach the conclusion it’s related to that, but I don’t see how that would explain your experiences either. Sorry I couldn’t be more informative!

CW_Jesse (Better Networking mod dev)
CW_Jesse (Better Networking mod dev)
4 days ago

If you do a true server/client model, then the server needs to do physics simulations. The Serverside Simulations mod does that, but it greatly increases the required server resources, both CPU and upload bandwidth. Eventually, the server becomes the bottleneck. Can we do better?

With the current Valheim method, you could theoretically have hundreds of players on a server as long as they were evenly distributed in different zones. The biggest problem is when the weakest link becomes the zone owner or the strongest link is overloaded.

The current model is actually better than it might appear at first glance… if done well. Simply having smarter logic for choosing who the zone owner is (based on CPU speed and upload bandwidth) would make a world of difference, and I could probably do that for Better Networking 3.0. Having smarter zones instead of a grid would be a further improvement, but that might not even be necessary.

There is a way to make money from this, but it’s not from making a publicly available mod: It’s either from becoming a server provider that outperforms (or is cheaper than) other services, which I could probably do, or using the fact you built a popular mod on your resume to get a job in tech, but I already have a job in tech. Most people don’t want to pay for mods. They’d sooner reverse engineer them than pay. But people do want to pay for servers. I appreciate your thoughts. If I do end up becoming a server provider, I’ll throw some money your way to pay for the idea. 😛

CW_Jesse (Better Networking mod dev)
CW_Jesse (Better Networking mod dev)
2 days ago

Yeah, cheating is, I think, the main reason to switch to a true server/client model. While you’re right to say Minecraft is more intense than Valheim all things being equal, not all is equal; switching to that model would also require me to optimize other parts of the game, not just the networking stack.

Clearly they spent their limited resources well; it’s a great game with great reviews. For every issue that bothers us, there are dozens of things that are so well done we don’t even think about them and are the reason we’re having this discussion in the first place, and as you said, we don’t know the story behind it. It probably would’ve been easier to just have a server/client model in the first place than to write code for switching zone ownership and such, which maybe means they realized it would be easier to switch to this hybrid model than optimize what was already written. Or maybe multiplayer was added late in the development cycle. Either way, we’re looking at the result of hundreds of good decisions, even if the end result isn’t exactly what we want. There’s a good chance it’s not exactly what they want, either.

Thank you for the great discussion! And thank you for these blog posts and discussions: they were one of my two original sources of education for making the first version of Better Networking in 2021, which is why you’re linked in the mod description.

KeenKeister
KeenKeister
1 month ago

When I try and Compile I keep getting error codes CS1001 and CS0246, any clues?

KeenKeister
KeenKeister
1 month ago

TY!

1 3 4 5