Revisiting Fixing Valheim Lag – Modifying Send/Receive Limits
Published: June 15, 2021 / Last Modified: December 3, 2023 / Gaming / 232 Comments
My Valheim Dedicated Server Castle
I’ve previously covered fixing Valheim’s dedicated server lag by modifying the dedicated server’s send receive limits. Since I wrote that article not long after launch though the game has been updated that the old method no longer applies. I had theorized in the comments of that article how to address this and today Sam R. confirmed that this method did indeed alleviate most congestion on the server in the places that the send/receive limits are known to bottleneck.
This updated guide will show some updated methods you can use to help reduce lag on your Valheim dedicated server!
Symptoms of lag on Valheim dedicated servers
The symptoms you are experiencing on the server can help narrow down what type of lag you are experiencing and what may be causing it. Lag caused by your CPU being too overloaded for example will have different symptoms than lag that is caused by your server hitting the maximum send/receive limit as an example.
By far the most common type of dedicated server lag seen in the early access Valheim release so far has been mainly due to the low send/receive limit if 64KB/s that is hard coded into the server. This means that when you do something like take your character a heavily populated area on the server it has to send all that information about all the buildings/players through an artificially limited very narrow 64KB/s pipe.
Symptoms of the send/receive limit type of lag include things like chests taking a very long time to open, sometimes 10-30 seconds or never opening while at the same time being able to see players walking around normally. This is because the other people’s X / Y / Z positions is a very small amount of data compared to all the items/names/attributes that are stored in the chest. The other players moving easily fits through the narrow pipe while the chest full of loot’s data chokes since it is much larger.
I’ll specifically lay out how to modify your server and increase this limit in the “Modifying your server’s send/receive limit” section below. Before that though let’s cover a more specific type of lag to watch out for (mostly by being avoided) related to terrain manipulation.
Avoid extensive terrain manipulation (for now)
Another commonly observed source of lag has to do with terrain manipulation. Some players have found that if they take a big chunk out of the ground in Valheim and make extensive terrain modifications that the area gets filled with separate “instances” of terrain for each modification people make. Think of the original ground as one big piece in the game files. Valheim has the very cool feature of letting you modify terrain by raising it/lower it/level it/etc. Each time you do this though the game has to store what you did as a new “instance” or a modification.
The way Valheim seems to work now is it renders that original piece and then the modifications afterward. Early reports suggest having a whole bunch of them in one spot seems to sometimes cause everyone’s in-game FPS on the server to drop as low as 40 when the clients normally are much much higher than that.
The best way to avoid this type of lag at the moment is to limit how much terrain modification you are doing in areas near your base. It is likely this will be addressed/eased in future patches. There should be some things the developer team can do to clean up/merge these instances or at least reduce their effect on the rendering process so it doesn’t drop the in-game FPS so dramatically.
At this time there is no options file for Valheim unfortunately. The only way to change it is to modify the server’s code and recompile the module that controls this.
Fortunately this is a lot easier than it sounds because the module we need to change is in .NET instead of something like C++ which would be much more difficult to do this with. I did not come up with this method, a very clever user on reddit (maximgame) shared it here and I want to give credit where credit is due!
Despite it not being too difficult these modifications are at your own risk. I haven’t experienced any problems so far but technically we are modifying the server’s code with the following procedure so understand that this isn’t an officially supported method and could have side effects or if you make a mistake could cause serious problems. Make sure you have backed up everything (your game world files especially) before you try anything!
Another important note is that it’s very likely you will have to make the changes again each update since the file we are going to modify is going to get replaced by updates. Until they add either a launch option or options file to officially change/control this property that is unfortunately unavoidable.
With all that being said if you understand the above and still feel comfortable let’s proceed!
Get dnSpy utility
We are going to use the utility dnSpy to modify the server code and recompile the module. The utility is free and is available here (download the Win64 build or dnSpy-net-win64.zip): GitHub dnSpy Official Page
Extract the archive and run the program dnSpy.exe from inside the folder you extracted. The dnSpy application will open.
Modifying server with dnSpy
We are going to go to the “File” menu in the top left and choose “Open” like this:
dnSpy – File -> Open Menu
Now you need to navigate to the Valheim dedicated server folder which is located in your “steamapps” folder. If you left all the options default when you installed it’s typically “C:\Program Files (x86)\Steam\steamapps\common\Valheim dedicated server\valheim_server_Data\Managed”.
We are looking for the file “assembly_valheim.dll” in that folder:
dnSpy – Selecting Valheim server assembly file
Select the file “assembly_valheim.dll” and click “Open” and the server’s assembly will load into dnSpy.
Once it has finished loading we are going to go to the “Edit” menu and click “Search Assemblies” like this:
dnSpy – Search Assemblies
For the search criteria enter “ZDOMan” like this:
dnSpy – Searching for ‘ZDOMan’
After the search completes you should see a list of results like the one above. Double click on one of these entries on the list and it should place you in the ZDOMan section of the assembly (seen on the left hand side of the screen):
dnSpy – ‘ZDOMan’ section of assembly
Perfect. We are now in the right place to make the changes. We just need to locate the line that has the send/receive limitation. To find that press Ctrl+F to bring up the find box (or choose “Edit” -> Find from the menu) and search for:
sendQueueSize
which should bring up the following lines:
dnSpy – sendQueueSize Variable
If you are having trouble finding the lines make sure you’re using Ctrl+F to search this time and not the “Search Assemblies” from earlier at the bottom of the screenshot, that’s a different type of search. Make sure you’ve disabled the case sensitive options (the Aa right below the box) or any other search filters. Your search query box should be in the top right of the screen like the above screenshot that has sendQueueSize in it.
Now right click on this line:
int num = 10240 - sendQueueSize
and click “Edit Class (C#)…”. Another window will open that will let you make changes to the file. We are going to modify ‘10240’ to a variable of our choosing. I made mine 30720 since after doing some conversion math that is the equivalent increase from my previous launch version guide that provided great performance and I kept them the same. If you followed my last guide this will give you the exact same performance as the old one did.
There are two instances of this variable. The second instance is 3 lines above the highlighted line we searched for and is:
if (!flush && sendQueueSize > 10240)
Modify both instances of 10240 to 30720.
Theoretically setting a limit too high could cause the server to swamp your internet connection if you have a large number of players on and there’s essentially no limit but there shouldn’t be any other effects of a very high number.
Once you’ve changed both numbers to the desired value we need to press the “Compile” button in the bottom right corner of this new window. This window will close and take you back to the previous one.
Saving the new assembly
It’s time to save our changes into a new .dll file. Choose “File” -> “Save Module…”:
dnSpy – Save Module as assembly_valheim_modded.dll
Change the file name to assembly_valheim_modded.dll. Don’t overwrite the original file just yet as we will want to take a backup and make sure that the server is closed before we do that. Press “OK” to write the new assembly_valheim_modded.dll file.
Backing up original file and replacing with our new one
Now it’s time to back up the original file and replace it with our new modded file. First make sure your dedicated server is closed otherwise the dll file will be in use. It is also a good time to make a backup of your world file just in case (see my Valheim Dedicated Server Backup Location / Guide for instructions)
Navigate in a file explorer window to your Valheim server directory where we saved the new assembly file (usually C:\Program Files (x86)\Steam\steamapps\common\Valheim dedicated server\valheim_server_Data\Managed).
First make a copy of the original assembly_valheim.dll file. You can just select the file and “copy” then paste a copy of it in that same folder and Windows will make a “valheim_assembly Copy(1).dll” file for you or you can back it up to a separate folder. Now remove the original file and rename your “valheim_assembly_modded.dll” file to the original file name of “valheim_assembly.dll”.
That’s it, you’ve now replaced the old assembly with the new one! Go ahead and start your server normally and the changes will have taken effect.
If you know of other Valheim lag issues that I didn’t cover here or are still having problems let me know in the comments and I will update this guide with other tips/tricks and try to answer any questions. It’s still a very new game and there isn’t very much out there for it yet. Have fun and enjoy!
I’m really not sure why mistlands has made this issue so much more prevalent. I’ve been playing on different servers hosted by myself and friends for over a year now and we have never had lag issues this bad before. I hope the devs change their tune about this being an ‘all the fault of those connecting to your server’ issue. I’m still playing with the same 5-6 people.
You’re right. It definitely has got worse and it makes sense why. They haven’t fixed the underlying problem but they’ve added a *ton* of new content/features/systems. Some people noticed the same thing about Hearth and Home and that era of updates. It was ever so slightly worse than launch. This jump is more dramatic to Mistlands though because of how much new stuff there actually is (including entire new systems).
There’s a whole mana/magic system now (I believe called Eitr). That’s all extra data and packets that have to go over an already taxed system. There’s more content, more data and more features but the underlying problem of how Valheim works with the clients controlling everything hasn’t changed.
It’s absolutely time to address it. It will only get worse if the devs add more features and more packets/data before fixing the multiplayer model. I am in complete agreement!
Yes, our entire team can confirm this, having completed Mistlands some time ago.
The general performance has gotten “significantly” (compared to the previous game version) worse again with mistlands-update starting with 3-4 players in one area.
It is as James has discussed with us several times here in the comments and after all this time with this knowledge, my observation and logical conclusion confirm this even more:
After each update there is more data to submit, while the server model has remained largely the same.
The guide here definitely helps make the multiplayer experience more enjoyable, but it doesn’t solve the problem. The problem is the code. Most noticeable is the positive improvement in the camp and when exchanging goods via containers, but also when fighting in groups, you notice a slight improvement – we compared it directly. It is definitely noticeable immediately when the .dll has been reset by an update. Although at first glance the procedure seems to bring little improvements, it is immediately noticed when it’s gone and you want it back immediately. So I recommand this to everyone.
What we also noticed are extremely annoying and heavy reload jerks since Mistlands, but this is another/new problem, which could be related to the world save.
I am shocked at how few players even complain about all this, how few notice it. Does anyone even play a co-op multiplayer game with multiple people? Have the standards of today’s generation of gamers dropped so low? Or are most players unable to recognize/understand these types of issues? Really, our entire group logged in, played for 5 minutes, and everyone noticed right away. Whatever is going on with the rest of the player base is beyond me.
You have to keep in mind:
We are talking about a problem that if I open a normal server through GPortal and play the game “normally” without mods (so as intended by the developer), that without modifying the .dll, it is no longer playable/or at least not enjoyable for all of us when there 4-5 players are in an area. The game has sold millions of copies and been successful for 2 years. You can’t explain that to anyone with normal common sense.
Anyway, we had our fun thanks to this fix.
I don’t know how I could have enjoyed – in particular mistlands now – if the guide didn’t exist
Thanks to James again.
Welcome back! Thank you so much for everything you’ve said here. It’s beautiful.
I only wanted to add that although things have been quieter in the comments on the post the traffic #s for this post are pretty insane:
I’ve never had numbers like this for Valheim. Not even launch and not even close. It’s the #1 article on my site even though people are saying hardly anything. It’s beating out new hardware releases that I’m ranking #1 on Google for!
Keep in mind that Google Analytics is blocked by ad blocking tools which almost all of you have installed (myself included, and it’s part of why there’s no ads on the site and all of my page values show as $0.00 from Google). Those numbers are vast underestimates as I can guarantee more than 50% of my audience has an ad blocker installed that will block Google Analytics. The only reason I use Google Analytics at all is so I can get some idea of what real people are looking at because when I look at the raw traffic numbers in the log files it becomes hard to tell which ones are bots as you can’t trust the user-agent the browser sends (and Google is very good at determining if they’re bots or not).
That is why it has been so quiet around here. I couldn’t be happier though. I’d assume that 95% of that traffic is successfully downloading Sketaful’s prebuilt binary. I can’t say that I miss walking people through how to do the modification all the time even though I love you guys. It’s nice to have a break and I will take it for as long as it lasts! I probably would have lost my mind if I had to help even a small fraction of the 1000+ people per week all do the modification. The number of people coming through is getting way too crazy for me to handle alone if everyone had to do this themselves one at a time so it’s a good thing this happened.
That means that roughly 1000 different players / teams a week look like they’re implementing this. They just don’t need any help thanks to Sketaful but they are coming here for sure (in numbers I’ve never seen before). There’s not much to say other than thanking him for putting it together but to be honest putting those kind of things together usually looks like this. Packaging binaries is a pretty lonely and thankless job despite how many people they will impact/touch. This site has much higher engagement than most and it can’t be any higher than 1 out of every 1000 people may actually be willing to leave a comment.
We can probably assume that I will only get 1 “hit” per team as once the person on the team that was technical enough to research / implement this fix found Sketaful’s link they would likely directly link their team members to his download page instead of my page (at least I would, I would link them directly to the file). That means I probably on average only get 1 hit per team of players. When you think about it this way 1000/week is a lot. That’s actually multiple thousands of people per week playing with and enjoying the tweak/improvements instead of just 1000.
I can see with my own eyes though on the traffic that this is actually doing better than ever despite being quieter and I think Sketaful is to thank for that aspect of it by just making it a lot easier for everyone. Despite us growing further and further from the Mistlands launch the traffic for this article is continuing to increase. It was only 700/week a couple of weeks ago. It’s getting *more* popular still which really surprises me. I thought we would start to see the drop off by now but it’s still accelerating as far as my traffic #’s go.
When I asked Sketaful if he was worried about any kind of legal trouble he said:
When it come to the legal aspect of uploading the files. Yes, it’s the property of Coffee Stain Studios but it’s not the actual game I’m uploading so either they care enough to tell me to pull it down or they don’t. I’m not hurting their business by hosting the files, I’d say I rather do the oposite. And Sweden. Lovely country. The effect a negative response from them would do is that the files will be removed (by me if they ask me to), they won’t press charges, and if, for some strange reason they will, they won’t win since they can’t prove it hurt them in any way.
Mad respect. I agree with him that he’s helping Coffee Stain Studios and not hurting them. As long as he feels like he’s operating safely within the laws of his country and nobody else (with a valid claim) has asked me to remove it / claimed any type of infringement I will continue to link to it. I’ve actually officially added a “Prebuilt Binaries Links” section to the main article today as he has kept up through several Valheim updates now.
Despite everything I’ve said here I really wish the average players would complain more about it to the developers to be honest. If they never heard the end of this issue I’m guessing they’d at least add an option for us to change this ourselves without modifying the .dll files. There does seem to be a weird level of acceptance in some ways. I wouldn’t have gone as far as Sketaful because that’s such a ridiculous thing to expect some unpaid community member to have to do vs. making the paid developers fix their own game or at least add the option but as I’ve said I’m very glad he did. It’s still quieter than I’d expect as far as people complaining about it on reddit / the rest of the internet though.
It’s also possible that maybe not as many people as we think are playing with groups of 4+ too but the game really is a lot more fun to play with others so I feel like they’re missing a lot playing with that low of a count. Then again this last update I just played with my wife so I guess even I fall into that category this patch as well.
Take care!
K S C
1 year ago
I’m a complete and utter moron, and DNspy is quite frankly far far beyond my ken. Can someone just, “make” a DLL that isn’t strangled down to dialup bandwidths, and upload it please? 🙁
It’s because every time Valheim receives an update (no matter how small) the old dll won’t work anymore with Valheim (and will often crash it depending on how extensive the changes were). If someone went through the trouble of packaging and hosting the file it would work for maybe 1-2 weeks before there’s some sort of minor patch at least that will create a new Valheim .dll. Trying to reuse old ones is going to crash it (if not immediately then intermittently).
You are asking someone “just make” an entire project to modify Valheim dll files and distribute them on the internet so you don’t have to learn to use the dnSpy interface? Who is going to pay for it? Why would anyone spend their time modifying dll files update after update after update for other people when they probably finish the entire game with their group in 2 weeks? That’s not to even mention that you’re talking about modifying Valheim’s code without their consent and then distributing it.
If they decide to go after you legally (and I have no idea if they would or not) for illegally modifying and redistributing their code you would get crushed unless you live in Russia or China basically. It’s a risk. Some companies are notoriously lawsuit-happy and sensitive about anything to do with their IP and code (and in many countries a lot of the patents/copyright/protections around intellectual property become useless/expire automatically if they aren’t actively defended in court or other people have been doing it without getting legally challenged before).
I’m honestly stunned this is the solution that seems reasonable and likely compared to just asking the Valheim developers to fix this issue. Did you think about asking the developers to add this to the Valheim dll? You know what would be great about that? The Valheim team has millions of dollars in resources and developers to program features like this into the game. Doesn’t that seem like a lot more reasonable of a course of action than expecting people to build some project around modifying this dll for the entire world update after update after update for nothing? Why did we buy the game then? Isn’t the money supposed to go toward that?
This whole article and workaround exist because the Valheim developers have a problem that needs to be fixed. The answer is definitely not spinoff mod projects to take care of this issue. That would definitely be the worst timeline where the Valheim developers absolutely refuse to do this and have officially announced it will never be done ever. Even the workaround in this is just a band-aid. They need to change the multiplayer model sooner or later and they seem to be aware of that. I really recommend any effort go toward encouraging the Valheim developers to just either add some options to do this or adjust their multiplayer model.
The forked Minecraft server community seems really similar to this idea. I definitely have seen communities and ecosystems like you’re saying but I just don’t think we’re there yet with Valheim. It is not unreasonable to ask them to eliminate the bottlenecks/lag that is happening related to other clients controlling the monsters. They’ve made a bunch of changes to the send/receive limits and related systems before. They know it’s an issue. If we continue to let them know that the fixes they’ve done have not been enough to take care of this I’m sure they will make further changes related to this system.
This is a fundamental flaw in the design of the game that needs to be addressed. There’s still a lot of bosses left before the game is complete and time to take care of this. Don’t let the developers off the hook on this issue because if they actually fix it the right way it will be 100x better than this mod ever was. A modded .dll distribution is still a terrible solution compared to the developers fixing the multiplayer model. That would be an actual game changer. This workaround is a band-aid in comparison if that makes sense. Hopefully that helps explain it!
I’m going to try to see if this fixes my problems. If it does, I’ll upload the .dll for you to http://sketaful.se/valheim/dll-fix/
That page will just list whats in it and I’ll upload every new one I make when I make it, so it might be a day or two late sometimes.
What James is talking about with the “effort” of compiling etc… Well, as long as it’s already being complied since I need it for my own server, it’s really just an two-click upload to share it. 😉
It’s almost no effort to compile it. Almost anyone who has followed the guide once before can make the mod in 60 seconds.
That’s why I just give out the instructions. It’s *maintaining* it that is hard. 1000x times harder than making the mod.
In fact it’s so easy that it’s honestly never been an issue before. There’s only a handful of people that legitimately do not have the technical skills to follow how to do this guide that I’ve seen post here in the comments before. Even in those cases though every group historically has had at least one person competent enough to follow a guide with screenshots. Just have that person make the mod for your group and send it out. That’s what everyone else has done.
Not everyone needs to be a technical expert so I’m not even shaming anyone who this is over their head. It’s not so hard though that one of the more tech-savvy people in your group shouldn’t be able to handle it. If they can’t follow the guide then I don’t understand how they can follow enough of this technical discussion to even understand what the issue is. They probably aren’t in which case replacing files willy nilly is probably not a great idea.
I have no issue linking to this whatsoever. If you defy the odds and maintain this I will only congratulate and thank you. I wonder though if it will be even a month before people are posting this link/file is outdated but time will tell! How long does it take to completely beat the game with a group? It has only ever taken me a couple of weeks. Do you want to keep doing this when you aren’t playing anymore? How long will that be?
You’re actually the first person who has tried. I don’t have another example I can link to of someone trying this. I’d imagine if it happened very often I’d have a graveyard of examples to post to but I guess it’s the Mistlands release that has pushed the popularity back up where people think they want to temporarily take on these projects (for the moment). There was nearly a year of silence on this post until Mistlands was released and in a month or two I imagine it will be silence again until the next major boss release.
That’s why it’s difficult for me to buy into these ideas. I literally didn’t hear a word from anyone about this mod or this game for a year. There wasn’t enough interest to generate even 1 comment about it. Now when I see the big surge from Mistlands interest it’s hard for me to forget that year of silence right before this and think everyone is just back and is going to actively play Valheim forever now! No. I’m expecting exactly the same thing as before. This interest will die off within a month or two and everyone will be gone until the next update.
I see the same thing in the Minecraft server community. Everyone wants to run and start a project while they’re actively playing. There’s literally thousands and thousands and thousands of Minecraft projects. I would say 99% of them are abandoned. Literally. Why? Because it’s really *hard* to maintain something when you aren’t actively playing anymore. It takes someone truly special to do that. And by special I mean that those of us who do it probably have some sort of mental health disorder to explain why we’re so weird (a little bit of OCD in my case, the same reason I like mowing the lawn or plowing the driveway when most people don’t, it’s stress-relieving and super satisfying to me to see everything go back to perfect order as it should be and maintaining projects can sometimes give this satisfaction in the same way as monotonous tasks like mowing the lawn).
These projects surviving the gap is unthinkable to me but I have a dozen or so projects on GitHub I’ve released for years so my perspective on how hard it is to launch and maintain these projects is probably different than most people. It really is maintaining it that is the hard part. I came up with this entire variation of the mod in a fraction of the time it would take to maintain a project like you’re trying to do where someone has to build and distribute Valheim’s files for every. single. release.
It’s actually horrifying to me that this route seems easier than convincing the Valheim devs to fix their own game. I can assure you it’s not easier and that it almost certainly won’t work over the long term unless you are truly one of those special ones that can maintain something they aren’t playing anymore. I wish I could somehow gather and combine all of the effort out there going into things like this and just channel all of that power into spamming the Valheim devs that they better not dare add a single new additional feature before they fix this issue. This is why this project wouldn’t be satisfying to me in the way a lot of my other projects are. I would just be angry every single time I updated it at the devs for not wanting to fix their own game (and a little at the players too for not straight up demanding it).
I congratulate you on your massive balls. I literally do not have the balls to commit to this project of maintaining this because I know what that commitment really means. I shudder thinking every Valheim update of all the comments I would get saying “UPLOAD THE UPDATED .DLL ALREADY”. I’m already having nightmares about it even though I never agreed to do it and am not at all involved! You are even hosting this on a Swedish domain and Coffee Stain Studios is a Swedish company. That is bold. Either that or you know they won’t / can’t do anything to you for some reason (as an American I can guarantee if they filed a lawsuit against me for doing the same thing I’d get crushed). Best of luck, truly, I am legitimately rooting for you despite my skepticism!
Otherwise I’ll be seeing all of you again just like I saw many of the same people from launch and Hearth and Home returning (like myself) to play the new content only to find the Valheim developers have STILL not fixed this issue and it’s exactly the same as when I published this. I hope that’s not the case next time. I really hope we’re not back next boss sharing our frustration with each other this still was never patched/fixed.
The files will be maintained atleast as long as I have my own server up since it’s basically just uploading what I already do every time it’s needed. It might take me like a day extra sometimes if I’m working, but beside that it will be there.
That’s pretty much how I maintain my projects that are “downstream” from Minecraft. I’ll update the projects / containers within a day or so of a new release.
Best of luck, I’m sure a lot of people have downloaded it already! I’m definitely seeing a lot less problems from people doing the modification so I’m assuming a lot of people are just downloading your release. Keep at it (as long as you stay cool with Coffee Stain Studios that is)!
When it come to the legal aspect of uploading the files. Yes, it’s the property of Coffee Stain Studios but it’s not the actual game I’m uploading so either they care enough to tell me to pull it down or they don’t. I’m not hurting their business by hosting the files, I’d say I rather do the oposite. And Sweden. Lovely country. The effect a negative response from them would do is that the files will be removed (by me if they ask me to), they won’t press charges, and if, for some strange reason they will, they won’t win since they can’t prove it hurt them in any way.
Btw. I live like 7km (around 4 miles) from their studio so I guess I just head over there and ask them why if they do. 😛
That makes sense, I’d love to visit Sweden someday! I’d imagine if you hear from them it would be because of what I was alluding to in my earlier posts (they can lose certain legal protections if they ignore things like this).
I’d love to think that instead of try any legal action they would just decide to add this in as an option in the “Options” menu (whether in the server, the client, or both). That would really be the best thing for everyone I think (no matter where they live or what connection they have). That way people can set this to whatever makes sense for the connections they and their players are using!
Yeah, I wouldn’t hold my breath. They’ve made an amazing game but it’s not very well optimized.
I wonder, Does it help to do this not only on the dedicated server but on the game itself?
…and is there any difference between the game’s assembly_valheim.dll and the one on a dedicated server or is it enough to use it in both?
Also. While at it, changing false to true in a couple of places let you use the odins dlc ;P
The client side mod is not as beneficial as the server-side mod but it has some really specific benefits. For example there are boss fights like the swamp boss fight that have a huge amount of entities on the screen. Doing the client side mod has significantly helped with scenarios like that because in those cases since we now know that the clients control the monsters we know that if there’s a lot of monsters that it has the potential to lag things simply because of the client’s send/receive limits instead of the server.
In other words you’re exactly right. This same mod can help on the client side. It’s not as helpful as the server one because the server is usually the one that is going to hit the send/receive limits. There are exceptions though such as major boss fights where since the clients control the monsters in those cases it can actually be the *client* lagging everything up. Those are the situations where the client side mod can help.
In fact for patk and a few other groups the client side mod was the *key* thing they needed to beat the game as a group. The server side one wasn’t enough because it was the boss fights that were really hurting them and it was due to the *clients* lagging. That’s why the client-side mod made a difference in those cases. It won’t have as general and noticeable of an impact but the situations where it does have an impact it can make all of the difference.
It would be distributed basically the same way. They just put this in the client .dll folder instead of the server .dll folder. Hopefully that helps, and thanks for continuing to maintain the binaries of the mod, people are loving it!
You will have to keep a view on the date of upload to figure out if it’s new enough.
FreezeLag
1 year ago
Hello everyone! Just wanted to add our experience if anyone else oversees this problem when it’s not working to modify the file if the serverhost has a linux system and sent the .dll file to a windows user to quickly modify it and send it back.
We tried to just send the “assembly_valheim.dll” file over and change the values before sending it back and overwrite the old file in the servers folder. This made alot of errors popping up in dnSPY when you tried to compile the file after the numbers were edited. Turns out you need to zip/rar the whole “Managed” folder with all the .dll files included and send it over for it to work as it seems to be looking for other dlls mentioned in the code when you compile it. This might be obvious to an experienced user but we totally overlooked it before we finally found out why it didn’t work. It did work like a charm after we had the whole folder on the computer editing the file though, afterwards we just zipped the whole folder down again after changing it and sent it back to the linux server system for replacment.
And thanks for guide James! Super helpful and easy to follow, cheers!
Great comment, thanks for leaving this! This makes sense. It sounds like what is happening is that dnSpy is resolving all the dependencies. That’s why you transferring the whole folder helped as now it’s resolving those correctly.
I definitely recommend anyone modifying Linux do the same. Transfer the whole folder. This should resolve the dependency errors. Thanks for sharing!
It’s the same for windows as well. It needs the whole folder to get it all correct. It looks for dendency files and if it doesn’t find them where it expects them it’s a ton of errors.
orbtl
1 year ago
Hey James, I’ve got another question. I was reading through an earlier thread you were responding back and forth in about how the netcode is really P2P, and a lot of the issues stem from the fact that tons of data regarding monster behavior and such have to be sent back and forth between the player that is “controlling” the monsters with their client, the server, and the other players. This got me wondering:
Should this data rate change you document in this article be made to the client application for all the clients playing, in addition to the dedicated server?
Say my client is the one in charge of the monsters and such and I have 5 friends in a dungeon with me. Isn’t my data rate cap on my client going to get hit and cause lag issues? Wouldn’t it help to raise that cap on my client, and not just the dedicated server?
Please let me know if I’m missing some key piece of information here, such as the data rate being set separately in code for the client than for the server, for example
Also check out patk’s post here. Most people seem to say there is still a benefit to doing this but that it’s not as drastic as when you do it to the dedicated server. It’s exactly how you say (patk’s post goes into this as well). If the client controlling the monsters is choking on the send/receive limit then that can absolutely lag the entire server and raising the send/receive limit can help with that.
In reality it’s a lot less likely to happen to the client controlling the monsters than the server but these are two separate things. The server has a huge benefit to raising it and the clients can have a marginal (and situational) benefit from raising it if controlling the monsters is causing that client to hit their send/receive limits. You can do one, the other, or both fortunately so it’s not one or the other and this is totally possible and can help.
An example of a situation where modifying the clients has been documented to help is the swamp boss battle. In this fight the swamp boss will constantly spawn a ton of minions and it can quickly add up to dozens (and they’ll be combined with all the other swamp creatures in the area which there will usually be a bunch running around interfering with the fight). This is a *LOT* more monsters for someone to control than a normal situation (or just about any other situation in Valheim).
We’ve seen the client side modification make the difference in that fight before (combined with the server modification). This situation makes sense too because it’s the situation where you will have the most monsters naturally in a fight together and it’s going to bump up against the send/receive limits of the client controlling them (when in normal situations this is not likely).
It’s a lot more complicated to get all of the players connecting to all do the mod or distribute the file but people have done this before for their group. It’s a lot more work but there’s a few people in the past who did have further improvements by modifying the client dlls (exactly as your suggesting). It’s just a question of whether it’s worth the additional effort and complexity as you will have to make the new file and distribute it to everyone each update. It has been done before and people do get tired of doing it over the longer term every single patch but it absolutely can have benefits according to others who have tried it.
Okay thanks, that makes a lot of sense. I’m currently experiencing a lot of lag issues even with the increased limits and it seems like something else is limiting the data rate, and I’m not sure what. Was hoping maybe you know. I increased the values to 10x instead of the 3x recommended here, and I noticed in our base with a few players connect I’m capped out at data sent: 150kb/s. I have symmetric gigabit internet so I can handle way more than that, so I tried upping the number to 100x and it still won’t send more than 150kb/s. Do you know if there is some other variable somewhere that is limiting my data sent so that my client (when the chunk master) can send more data to handle all of the physics going on?
That is basically patk and Greg’s journey from page 1 of the comments. Read patk’s thread here:
Thanks for this guide, once we implemented this guide with a lot of mods installed our send and receive caps out at 150kb/s and when this happens we get massive desync.
The valuable content in this article is actually the comments and not the guide. The guide is just the easiest and most fruitful mod to do. It’s just a taste basically! If you want to go deep on this though yes you can go much much further and we already have/did back then pretty thoroughly including lifting the 150kb/sec cap.
If you go back from comment page 1 and just read through them you will know everything you need to know for sure. It’s from a year ago and that will save you a lot of time as you’re kind of repeating patk/Greg’s journeys and they documented the journey very well (making it easy to repeat). A bunch of others have chimed in as well and this has all been thoroughly investigated/tested on the 150kb/sec limit you’re seeing (which can be changed, yes). In other words they already took the hits for you on this and left a record of it so you don’t have to start from square one fortunately.
There are a bunch of pitfalls to what you are trying to do. For example if everyone’s send/receive limits don’t match there is massive packet loss. Definitely read through that whole thread though from patk as he did this and outlines exactly what to do for the 150kb/sec problem and some of the pitfalls to watch out for. If it’s all done right people have had successes for sure.
The short answer is that yes there is more to the send/receive limits than what the guide is modifying. There’s additional code in there in those comments on how to do it but if you do it everyone’s needs to match that is playing or it will make things very very strange. It’s really hard to describe it better than that though as it’s all written down on page 1 of the comments and will describe it better than I am here. Hopefully that helps!
Russ
1 year ago
This doesn’t seem to work for my Linux Dedicated server. The server crashes shortly after starting. Do you have any hints on how to properly edit this file for a Linux Dedicated server ?
Everyone who has had trouble with this in the past was trying to edit the Windows version of the file and use it on Linux. Even though the game is in .NET these files are actually very different from each other and not compatible. The guide will basically work exactly the same but you will need to physically retrieve the Linux version of the files from the machine, edit it, and then put them back.
In all cases they were not modifying the right files. If you are trying to modify the Linux files from a Windows machine (which is what literally everyone else who has tried this was doing that I can recall) then you actually need to get the files from the Linux machine and transfer them to your Windows machine and then make the modifications.
The files are different between the different platforms. The crashes tell you for sure you are using one for the wrong platform/architecture. The modifications we are making will not crash it even if you turn the number way, way higher than you should (it will just swamp your connection). The crashes basically mean that the wrong files / platform / version is being used somehow (almost certainly the Windows version).
I’d definitely try pulling the right files from the Linux machine directly and editing them. Make sure it’s not using dnSpy’s cache and is actually opening the file that you think it is. We just had an issue yesterday where dnSpy was caching older versions of the file despite them being positive they opened the newer one (and I believe them). A lot of reverse engineering tools work this way and have a cache/database and it’s not always obvious that is how it’s working and in fact can sometimes be quite difficult to refresh unless you’re pretty experienced with the products. It’s very quirky and you will probably need to completely close all existing edits and files out. Double and triple check this because a crash indicates the wrong files are being edited for sure.
It’s fine to modify them using Windows but the process is you need to get the file from the Linux machine, edit it on Windows and build the new version then take it and put it back on the Linux machine. This has been done before a few times but it’s extremely important to use the Linux files as the Windows ones will definitely lead to this exact crash. .NET apparently doesn’t work like Java where you can run the same .jar on every platform. The .dll files are actually different between the two platforms (although the code inside dnSpy will look identical, it’s probably only the dependencies/linking/etc. that is different).
Thanks for the info. I pulled the files directly from the Linux server, as it is also in my office. But I initially had the Windows files open from my own installation on my PC, where I was editing, so I think I was a victim of the cached files. I appreciate your insights. Thanks for the guide!
No problem at all, I suspect it’s the cached files issue as well. Shockingly orbtl’s dnSpy copy managed to pull up a cached version from launch after all of this time. I’m sure those files are long gone and that they must have only existed in dnSpy’s cache at this point (which is probably why they understandably thought this was a recent copy of the current dll on the system).
Thanks for letting us know what it was and ended up looking like. This will help others as well especially working with the Linux files. Enjoy and take care!
orbtl
1 year ago
Hello! Thanks for this wonderful guide. I remember following these modifications over a year ago and they were very helpful. However I have just returned to the game and it seems like the code has changed. Do you know if there’s a way to still make these changes with the changes to the netcode (or if they are still necessary?). The SendZDOs method that contained the code you modified appears to have been rewritten and no longer contains that hard-coded integer value you were modifying. Here is what I see in the code now:
private int SendZDOs(ZDOMan.ZDOPeer peer, bool flush)
{
if (!flush && peer.m_peer.m_socket.IsSending())
{
return 0;
}
float time = Time.time;
this.m_tempToSync.Clear();
this.CreateSyncList(peer, this.m_tempToSync);
if (this.m_tempToSync.Count <= 0)
{
return 0;
}
int num = this.m_dataPerSec / 20;
ZPackage zpackage = new ZPackage();
ZPackage zpackage2 = new ZPackage();
int num2 = 0;
for (int i = 0; i num)
{
break;
}
}
}
if (num2 > 0)
{
zpackage.Write(ZDOID.None);
peer.m_peer.m_rpc.Invoke("ZDOData", new object[]
{
zpackage
});
}
return num2;
}
Can you try following my old guide and see if you can find the m_dataPerSec variable? This is very strange though that they would completely change it back to the old method. It’s back to m_dataPerSec according to your code. This entire article was written because that variable was completely removed from Valheim at the time but my original guide for this style of code is still up. This was how it worked at launch actually.
Unless this is a copy installed from hard media that hasn’t been updated (did they even ship this game on hard media?) or I can’t even imagine how this could happen unless they actually have changed it back in the current version (unless you restored files from a really old backup over the top of a newer install or something).
Are you perhaps editing the client .dll instead of the dedicated server .dll? I’m not sure they ever changed the client away from m_dataPerSec. Modifying each client can have marginal benefits but testing from back in the day showed that 90% of the benefit seems to be boosting the send/receive limits on the dedicated server (which is acting as a relay for everyone).
It makes sense that pretty much all of the benefits of raising the send/receive limits will be on the server since the send/receive limit can basically be thought of in bytes per second. The server is relaying for *all* of the clients so it’s going to bump up against the ceiling way before any clients will simply due to the fact that for any of the clients to talk to each other (which they do in Valheim even going as far as controlling each other’s monsters) it has to be relayed through the dedicated server. It has a lot more to send/receive as the server than any client ever will which is why it hits the send/receive limits first.
Either way you should be covered by following the old guide instead!
Thank you so much. I guess dnSpy keeps a cache of files you have opened previously or something? Once you pointed out that this was actually the old code, it made me realize this must be the case. I made sure to hard close everything in dnSpy and re-open the .dll again and the new code showed up as it does in this article. Sorry for my mistake and thanks so much for quickly helping me figure out what was wrong!
No worries at all, that actually makes sense that it could be something like that! It’s not uncommon for other reverse engineering tools (Ghidra / IDA Pro come to mind) to similarly cache/import a dll/exe/etc. It would almost be hard to find some of these files if you were looking for them intentionally as that has been gone from the server for quite a while.
I’m glad that helped and thanks for leaving the comment! Surely there are others that will be coming back to try Mistlands that will probably have the same thing happen and hopefully this saves them some time.
Enjoy and take care!
Patrex
1 year ago
Hello, im running a dedicated server and i have some of those sync/lag issues. We are 4-6 people playing constantly.
Sometimes doors and chests take a long time to open. The ennemies/animals look like they are teleporting.
Do you think its a CPU/RAM issue or more like the problem with server’s send/receive limit ?
Is this post still valid in november 2022 ?
Yes, that definitely sounds like the normal issue. The amount of players is still about right as well. I’ve started playing the Mistlands as well with my wife but I haven’t been encountering the lag with only 2 people on.
People have still noticed some benefits doing a process like this but the problems are mainly due to scaling with Valheim. Modifying the send/receive limits can help reduce/alleviate symptoms, yes, but it can’t eliminate the problem. The amount of stress on the server going from 4 to 6 is quite substantial. It definitely doesn’t increase the load in a linear way to add players. It’s exponential because it has a heavy peer-to-peer component.
That also means the lag can come from other players. It doesn’t even necessarily need to be that someone has a bad connection to the server. The players having a bad connection to *each other* will also lag the server. This is why it’s really complicated and why this is only a tool that can help and not really a solution. The solution is Valheim would need more scalable network code (probably a rewrite on the network code).
I’d say the state of this issue and the potential fixes are exactly the same as they’ve always been. The send/receive limits are one tool you can use to try to reduce the lag caused by Valheim’s scaling issues. It won’t fix them now and if you read the older comments from back in the day it didn’t fix it back then either. Did it make things marginally better though? Yes, and jacking the send/receive limits up higher today can help sometimes for the same reason it did back then.
It depends on exactly what the issue is causing the lag on your server. This issue is absolutely identical to launch in my opinion. Nothing has changed. Valheim’s scaling issues will get you into trouble with the send/receive limits really quick today on any server with 4-6 people as it will overwhelm your client’s networking capabilities even if you are running it with a dedicated server in a data center.
A lot of the people who commented on this originally had enterprise equipment. It doesn’t matter. The lag isn’t necessarily coming from the server but from the process of all the clients exchanging P2P data with each other (and things in the game basically stop until they’ve all spoken and come to an agreement about what is going to happen on everyone’s screen, it’s a silly system).
An example of a successful “fix” from this would be people being able to do a boss fight without lagging so bad they couldn’t do anything. Some of the biggest success stories using this method are exactly that. They nibbled around the edges enough so they could get through one fight without being miserable. In some cases that was honestly enough to make people and the group very happy as it was that one boss fight that was holding them back from beating the entire game as a group.
It was simply a matter of nibbling at the edges of the networking stack to try and make up for the multiplayer design of the game which you can still do today. It will never be able to support more than the amount of players you are saying though without a networking code rewrite. It only gets worse if you went to 7 or 8. Much worse.
Understand though that there was never what people would call a true “fix” where you fix some setting and never have to worry about this again. That’s not possible without a network code overhaul and essentially a completely different system that drops all the P2P stuff. It’s more like a mod to try and tilt the scales in a slightly unplayable situation to playable by basically manipulating the network stack to use inappropriately large send/receive buffer sizes that would be completely unreasonable in any other situation because it means they chose a poor design for their networking model. In retrospect though I should have never used the word “fixing” in the title as the only people that can truly “fix” this are the Valheim developers.
Normally as a developer this is the part where you realize you need to change your networking model because your original choice won’t scale and making the send/receive buffers enormous is a hack that won’t fix anything and only buys you some more breathing room (in the form of maybe having 1 or 2 additional people online) before the problems hit.
This type of networking is what things like the original Xbox Live or older Playstations used. This is why most of the older consoles only supported a max of 8 players (and sometimes not even that). They used exactly this type of networking and a lot of those console games would lag with the same scaling issues as Valheim if you actually got 8 people in the lobby (apart from the nightmare of the host logging off and having to migrate to a new host and a bunch of other fun P2P stuff if people remember that).
I would venture to guess whoever wrote the multiplayer code from Valheim must come from a game background that used this type of networking and I don’t understand why you would choose a P2P model when you literally have/offer a “dedicated server”. In Valheim the dedicated server really just means it stores the map file. It’s still a P2P game even with a “dedicated server” which yes makes no sense but that is what it is.
Although Valheim’s developers say they’ve raised the send/receive limit in the code all that means is they implemented this same hack that I’ve wrote about in this article into their code instead of actually fixing the underlying problem. Instead of overhauling the networking code like I’ve always said is what needs to happen instead they implemented my hack into their code and called it good which has led us to today where sure enough this still isn’t fixed.
You can still turn the setting higher than them though. In the end this is a number in code and you have to choose how big it is. My change I wrote about here just makes that number bigger than they did. You can still make it bigger than they did today if that makes sense. It does mean you’ll probably get diminishing returns out of raising it though. They already raised it at least once but again you can always raise it higher.
You get to a point though where your send/receive limits are so high you will hit the limits of your player’s connections. It’s game over at that point. There’s nothing else to milk or tap. You have to rewrite the game’s multiplayer code at that point as it’s absurd that these players can likely stream 4K video with ease but can’t play Valheim with 6 players. You can watch 4K video don’t have enough bandwidth to play Valheim with 6 players using this model which is kind of hilarious to think about and shows how bad the scaling really is (and how much data we are talking about that is getting duplicated and sent between all these different P2P clients). That’s a problem with the game itself. A structural problem (just to explain).
Maybe a second wave of popularity bringing light to this issue will prompt them to finally rewrite the multiplayer networking code? That’s what needs to happen though and hopefully my super long-winded explanation provides some light on why this happens at least.
Thanks for the detailed reply.
Yesterday we were 9 on the server and i noticed that the problem is only when we are in the same sector of the map.
If i take a portal to another continent far away, i dont have any sync issues.
But as soon as i come back to our village, i can see the problem (Mobs teleporting, delay on doors, chest etc)
Only with 3-4 person in the village and its very intense, cant even fight a group of Greydwarves
Its kinda sad since the point of multiplayer is to play in group with friends…
I also found a post from one of the Devs who said the game is not using a P2P network :
I’ll still try your fix tomorrow and we will see if its a little better at least, but i hope they fix the code soon…
That developer is playing games with the definition:
Thanks Richard. I understand the nuance but it was easier to say P2P than explain the nuance. I fully appreciate that it's complicated and appreciate the response.
I think there are two classes of users,
those with DSL who are limited by the transmit rate. It's very close to DSL limits
those with lossy networks.
The former can really only be addressed by limiting the bandwidth. short of some compression scheme the one obvious approach would be to halve the update rate. Makes for some choppy behavior but it would work. There are some other ideas but get much more complicated, e.g. balance responsibility for objects by network capacity but that risks clobbering machines with great internet and poor processing.
The latter can be addressed in small part by posting a sticky here with instructions for how to diagnose and correct most forms of loss. I've seen that done in other games and would be happy to help contribute. It has the added advantage of helping users QoL beyond the game. After that there's not a lot one can do for lossy traffic with this server model.
Don’t be fooled by this. The developer is saying technically it’s not a P2P game and that it’s more “nuanced”. Do you know what that nuance is? It’s what I spoke about in my previous post. It works like a P2P game even if it technically isn’t one. It might as well be P2P since it has the same restrictions and limitations. Yes, we know it’s technically not a P2P game, it has a dedicated server. That dedicated server is fake though (or at the very least calling it a dedicated server is misleading). It doesn’t work like the dedicated server in other games and is a glorified map storage computer.
On other games the server is controlling literally everything (in most games actually) and it generally does not care what the clients are or aren’t saying (nor does it trust it for security reasons). Obviously there are limits to this like if someone completely lags out they are going to just stand there on the server until they drop. The whole server doesn’t stop though and wait for clients to report what they’re seeing on their screen like it does in Valheim (nor should it, this is how people make very nasty exploits). People’s game clients are still running the show in Valheim even with a dedicated server and it’s such a strange and inappropriate meshing of these two types of multiplayer models.
Look what the developer said here:
It is however pretty sensitive to one player having a bad connection. If player-A is controlling the monsters in the area has high ping to the server then player-B will also notice this
This is what I’ve been trying to explain. The Valheim clients are controlling the monsters! You read that correctly! No, this is not how other multiplayer games work. They said this lets them do a lot more things like physics client side. As someone who has developed multiplayer games before and have released several of them on my GitHub I cannot believe they are having the CLIENTS control other people’s monsters in MULTIPLAYER! WHAT?! This is exactly why it lags and is essentially a P2P game even though they are quibbling with the definition. He just said it. Player A’s client is controlling player B’s monsters! In multiplayer! When they have a dedicated server! Why?!
If we are going to talk about nuance though they have changed some things to not rely on the clients so heavily. For example now in Valheim when the server is lagging you may see people moving now. That didn’t used to happen. It’s because they moved that and a couple of other things out of the main single-cored thread that does all of the game logic. Watch for this when you are playing. You may see people move around and not be able to do anything else.
That’s because 99% of everything in the game is still single-cored and all of that will stop. Only movement and I think like maybe opening doors or something got moved outside the main single-cored game logic thread? It wasn’t much though. This only makes it look like the server is still operating because people are technically moving around but you won’t be able to use any abilities or do anything if that makes sense. This is just a visual trick though that doesn’t solve the problem. It just makes it look less jarring when people keep moving generally in the direction they were going instead of just stop dead in their tracks but the server is in fact still stopped dead in it’s tracks for all of the other functions that are not broken off the main game logic thread.
This is so easy to disprove. How many of you have DSL? I bet you probably have something greater than DSL and are still experiencing this issue. Can you play other multiplayer games? Have you ever experienced anything like you have in Valheim before on DSL or otherwise? No? Other games don’t work this way? Exactly. Valheim definitely is the issue for sure here!
Look at the sentence at the very end:
there's not a lot one can do for lossy traffic with this server model
BOOM. There it is. This person responding and I completely agree and reached the same conclusion. There’s not a lot one can do with this server model. That is my *exact* argument. The Valheim server needs a different multiplayer model to solve this problem!
Honestly my takeaway from that developer post is that they knew all the way back then that this needs a new multiplayer model to solve this problem. I think I’m more annoyed after reading it and was more content believing that they just didn’t really understand the problem or something like that. They actually explained some things I didn’t dare to say directly such as Player A is controlling Player B’s monsters.
To be fair to the developers the approach they are using has it’s advantages. You know how they mentioned physics? A common problem with dedicated servers in other games is your hits not registering. Valheim is pretty forgiving with hit registration (although your hits may take a long time to appear if the whole thing is all lagged up, they *will* appear eventually). Your hits will always register because they’re cheating. They aren’t calculating the hits on the server. They are literally taking your client’s word for it and the decisions are made based on what your client says. I couldn’t disagree with their decision more to use a model like this though as the downsides far, far, far outweigh slightly better hit registration (which shouldn’t be an issue anyway on a well configured dedicated server).
I knew it worked something like that but there it is straight from the developers. This is why it’s a P2P game no matter what they say technically qualifies it as not P2P. If other players are controlling other people’s monsters (and not the server) that is P2P. Period. And with all the disadvantages that come with it such as one player being able to lag an entire dedicated server (which should be impossible).
It really doesn’t make it any less P2P that they wrap all of this through a dedicated server. It seems like they mean that it doesn’t work like a torrent does where you have a bunch of connections with each other. No, this system is actually worse than that. It has to all pass through the dedicated server who would serve as a relay (or the glorified map storage machine I alluded to earlier) sending these back and forth between clients instead of them talking directly.
That’s actually less efficient than P2P in a lot of cases. It’s P2P but with the additional step of everything having to be relayed through the dedicated server instead of sent directly from one peer to another. At the very least it doubles everyone’s ping effectively and that’s just from the math of having to go to the server, then go to the player, then go back to the server and then back to the original player again. In P2P it would look something like packet -> response. In this model it’s packet -> server -> player destination -> response sent back to server -> response sent from server back to original client. That’s a lot more steps for even a ping / pong.
It would be better if they picked one or the other. Go P2P or go dedicated server. At least then we would get the advantages of either P2P or a dedicated server instead of the disadvantages of both.
I honestly think now would be the time for the Valheim developers to do this. They just got Mistlands out and from what I’ve seen so far there’s quite a bit to do. There’s still several more bosses/biomes to go before the game is completed. They can be absolutely forgiven for choosing the wrong multiplayer model initially as they were a very small team at launch. I hope they don’t wait for any more bosses to address this issue. Hopefully that clarifies a bit more of what is happening here!
I upgraded our server to Mistlands too. We are now up to 6 players and tried playing without this fix at first within the new update.
We encountered still the same problem, nothing has changed with regard to the lag problems.
My hope was that when Mistlands drops for the console release, the main focus of the developers will be on the lagfix and that was also the reason for the long waiting time. But I was wrong. I admit I am a little disappointed what they have been doing all this time aside from implementing a few new models (/no hate).
Anyway, I applied all changes from this fix again on my server assembly_valheim.dll and got the same results as before the update. A lot more playable/enjoyable especially when player 4 and 5 hits the same area and I would recommand this fix to everyone -> Big thanks to James. I repeat: The lag problem is still! exactly like James figured it out on comment page 3 in our discussion.
I also recommand for all server hosts to ensure that every client has done its FPS boost (boot.ini configuration, disabled steam overlay, plays with “-window-mode exclusive -screen-fullscreen” and sets Task Manager priority to high). In case there are players on the server who are not playing on a high end device, it’s better to avoid extreme FPS drops.
Remember: The lag is as bad as the weakest link for all players in one area.
Thank you so much for verifying this! I had a bigger group I was playing with the last time but this time it’s just my wife and I playing at the moment so I didn’t have enough players to test this.
We know for sure that they haven’t changed the multiplayer model though so I knew that for all the technical reasons that we’ve covered this issue shouldn’t have changed (or the potential solutions).
I wasn’t sure what kind of diminishing returns you’d see as the Valheim team had also raised their own limit but as I’d outlined before at the end of the day they have to pick a number. They are never going to pick as high of a number as you will because too high of a number will completely overwhelm people’s connections.
They’ll never be able to safely set it for the entire world as high as you could set it yourself (with the caveat that you may swamp your own or your player’s connections going too crazy with it). They have to take into account computers and connections that are far less capable than anything you or your group are likely to be willing to use when it comes to setting a worldwide global release value.
It’s just honestly too bad they still haven’t made this a configurable option. You have to directly modify the .dll files still to this day to change this unfortunately. On many other game servers (especially Steam which Valheim has integration with even) you can actually just set the send/receive buffers value right in a config.txt file or something like that depending on the game. In Valheim this is still a hard-coded value.
Situations like this though are exactly why many other game servers let you configure this. You can’t ship a game with a perfect preset value for the whole world when it comes to networking or even hardware (which is why there are so many options). It’s almost always going to end up conservative/too low for the vast majority of likely players in order to make the game playable on the really poor connections/computers out there.
I’m very grateful to you for testing and reporting back that this workaround is still helping and showing noticeable impacts. Enjoy and take care!
It’s basically a P2P game with one server between that only have one job, to register all the changes made to the map when you’re offline. You could basically have it as a P2P game which keeps the latest mapchanges on OneDrive so everyone downloads that when starting their games. 😛
Yes, you’re exactly right and that’s a pretty funny analogy to think about this with. It absolutely might as well be a cloud hosted map. It wouldn’t impact the mechanics of the game at all since we know and have discussed at length the client-controlling aspects of the monsters and other parts of the game.
In fact if you go in and back up the server data there is literally only a map file to back up. All of the players profiles are hosted on their accounts / their computers. It is just the map file and the chest contents pretty much. Beyond that the server is only serving as a relay between the clients (less ideal than P2P even honestly since in P2P it’s a direct link vs. a middleman “dedicated” server)!
I’m really not sure why mistlands has made this issue so much more prevalent. I’ve been playing on different servers hosted by myself and friends for over a year now and we have never had lag issues this bad before. I hope the devs change their tune about this being an ‘all the fault of those connecting to your server’ issue. I’m still playing with the same 5-6 people.
Hey Graelynde,
You’re right. It definitely has got worse and it makes sense why. They haven’t fixed the underlying problem but they’ve added a *ton* of new content/features/systems. Some people noticed the same thing about Hearth and Home and that era of updates. It was ever so slightly worse than launch. This jump is more dramatic to Mistlands though because of how much new stuff there actually is (including entire new systems).
There’s a whole mana/magic system now (I believe called Eitr). That’s all extra data and packets that have to go over an already taxed system. There’s more content, more data and more features but the underlying problem of how Valheim works with the clients controlling everything hasn’t changed.
It’s absolutely time to address it. It will only get worse if the devs add more features and more packets/data before fixing the multiplayer model. I am in complete agreement!
Yes, our entire team can confirm this, having completed Mistlands some time ago.
The general performance has gotten “significantly” (compared to the previous game version) worse again with mistlands-update starting with 3-4 players in one area.
It is as James has discussed with us several times here in the comments and after all this time with this knowledge, my observation and logical conclusion confirm this even more:
After each update there is more data to submit, while the server model has remained largely the same.
The guide here definitely helps make the multiplayer experience more enjoyable, but it doesn’t solve the problem. The problem is the code. Most noticeable is the positive improvement in the camp and when exchanging goods via containers, but also when fighting in groups, you notice a slight improvement – we compared it directly. It is definitely noticeable immediately when the .dll has been reset by an update. Although at first glance the procedure seems to bring little improvements, it is immediately noticed when it’s gone and you want it back immediately. So I recommand this to everyone.
What we also noticed are extremely annoying and heavy reload jerks since Mistlands, but this is another/new problem, which could be related to the world save.
I am shocked at how few players even complain about all this, how few notice it. Does anyone even play a co-op multiplayer game with multiple people? Have the standards of today’s generation of gamers dropped so low? Or are most players unable to recognize/understand these types of issues? Really, our entire group logged in, played for 5 minutes, and everyone noticed right away. Whatever is going on with the rest of the player base is beyond me.
You have to keep in mind:
We are talking about a problem that if I open a normal server through GPortal and play the game “normally” without mods (so as intended by the developer), that without modifying the .dll, it is no longer playable/or at least not enjoyable for all of us when there 4-5 players are in an area. The game has sold millions of copies and been successful for 2 years. You can’t explain that to anyone with normal common sense.
Anyway, we had our fun thanks to this fix.
I don’t know how I could have enjoyed – in particular mistlands now – if the guide didn’t exist
Thanks to James again.
Best regards/
Hey Gesper,
Welcome back! Thank you so much for everything you’ve said here. It’s beautiful.
I only wanted to add that although things have been quieter in the comments on the post the traffic #s for this post are pretty insane:
I’ve never had numbers like this for Valheim. Not even launch and not even close. It’s the #1 article on my site even though people are saying hardly anything. It’s beating out new hardware releases that I’m ranking #1 on Google for!
Keep in mind that Google Analytics is blocked by ad blocking tools which almost all of you have installed (myself included, and it’s part of why there’s no ads on the site and all of my page values show as $0.00 from Google). Those numbers are vast underestimates as I can guarantee more than 50% of my audience has an ad blocker installed that will block Google Analytics. The only reason I use Google Analytics at all is so I can get some idea of what real people are looking at because when I look at the raw traffic numbers in the log files it becomes hard to tell which ones are bots as you can’t trust the user-agent the browser sends (and Google is very good at determining if they’re bots or not).
So why aren’t the comments exploding? I think it’s simply that everyone is downloading Sketaful’s prebuilt one here.
That is why it has been so quiet around here. I couldn’t be happier though. I’d assume that 95% of that traffic is successfully downloading Sketaful’s prebuilt binary. I can’t say that I miss walking people through how to do the modification all the time even though I love you guys. It’s nice to have a break and I will take it for as long as it lasts! I probably would have lost my mind if I had to help even a small fraction of the 1000+ people per week all do the modification. The number of people coming through is getting way too crazy for me to handle alone if everyone had to do this themselves one at a time so it’s a good thing this happened.
That means that roughly 1000 different players / teams a week look like they’re implementing this. They just don’t need any help thanks to Sketaful but they are coming here for sure (in numbers I’ve never seen before). There’s not much to say other than thanking him for putting it together but to be honest putting those kind of things together usually looks like this. Packaging binaries is a pretty lonely and thankless job despite how many people they will impact/touch. This site has much higher engagement than most and it can’t be any higher than 1 out of every 1000 people may actually be willing to leave a comment.
We can probably assume that I will only get 1 “hit” per team as once the person on the team that was technical enough to research / implement this fix found Sketaful’s link they would likely directly link their team members to his download page instead of my page (at least I would, I would link them directly to the file). That means I probably on average only get 1 hit per team of players. When you think about it this way 1000/week is a lot. That’s actually multiple thousands of people per week playing with and enjoying the tweak/improvements instead of just 1000.
I can see with my own eyes though on the traffic that this is actually doing better than ever despite being quieter and I think Sketaful is to thank for that aspect of it by just making it a lot easier for everyone. Despite us growing further and further from the Mistlands launch the traffic for this article is continuing to increase. It was only 700/week a couple of weeks ago. It’s getting *more* popular still which really surprises me. I thought we would start to see the drop off by now but it’s still accelerating as far as my traffic #’s go.
When I asked Sketaful if he was worried about any kind of legal trouble he said:
When it come to the legal aspect of uploading the files. Yes, it’s the property of Coffee Stain Studios but it’s not the actual game I’m uploading so either they care enough to tell me to pull it down or they don’t. I’m not hurting their business by hosting the files, I’d say I rather do the oposite. And Sweden. Lovely country. The effect a negative response from them would do is that the files will be removed (by me if they ask me to), they won’t press charges, and if, for some strange reason they will, they won’t win since they can’t prove it hurt them in any way.Mad respect. I agree with him that he’s helping Coffee Stain Studios and not hurting them. As long as he feels like he’s operating safely within the laws of his country and nobody else (with a valid claim) has asked me to remove it / claimed any type of infringement I will continue to link to it. I’ve actually officially added a “Prebuilt Binaries Links” section to the main article today as he has kept up through several Valheim updates now.
Despite everything I’ve said here I really wish the average players would complain more about it to the developers to be honest. If they never heard the end of this issue I’m guessing they’d at least add an option for us to change this ourselves without modifying the .dll files. There does seem to be a weird level of acceptance in some ways. I wouldn’t have gone as far as Sketaful because that’s such a ridiculous thing to expect some unpaid community member to have to do vs. making the paid developers fix their own game or at least add the option but as I’ve said I’m very glad he did. It’s still quieter than I’d expect as far as people complaining about it on reddit / the rest of the internet though.
It’s also possible that maybe not as many people as we think are playing with groups of 4+ too but the game really is a lot more fun to play with others so I feel like they’re missing a lot playing with that low of a count. Then again this last update I just played with my wife so I guess even I fall into that category this patch as well.
Take care!
I’m a complete and utter moron, and DNspy is quite frankly far far beyond my ken. Can someone just, “make” a DLL that isn’t strangled down to dialup bandwidths, and upload it please? 🙁
Hey K S C,
It’s because every time Valheim receives an update (no matter how small) the old dll won’t work anymore with Valheim (and will often crash it depending on how extensive the changes were). If someone went through the trouble of packaging and hosting the file it would work for maybe 1-2 weeks before there’s some sort of minor patch at least that will create a new Valheim .dll. Trying to reuse old ones is going to crash it (if not immediately then intermittently).
You are asking someone “just make” an entire project to modify Valheim dll files and distribute them on the internet so you don’t have to learn to use the dnSpy interface? Who is going to pay for it? Why would anyone spend their time modifying dll files update after update after update for other people when they probably finish the entire game with their group in 2 weeks? That’s not to even mention that you’re talking about modifying Valheim’s code without their consent and then distributing it.
If they decide to go after you legally (and I have no idea if they would or not) for illegally modifying and redistributing their code you would get crushed unless you live in Russia or China basically. It’s a risk. Some companies are notoriously lawsuit-happy and sensitive about anything to do with their IP and code (and in many countries a lot of the patents/copyright/protections around intellectual property become useless/expire automatically if they aren’t actively defended in court or other people have been doing it without getting legally challenged before).
I’m honestly stunned this is the solution that seems reasonable and likely compared to just asking the Valheim developers to fix this issue. Did you think about asking the developers to add this to the Valheim dll? You know what would be great about that? The Valheim team has millions of dollars in resources and developers to program features like this into the game. Doesn’t that seem like a lot more reasonable of a course of action than expecting people to build some project around modifying this dll for the entire world update after update after update for nothing? Why did we buy the game then? Isn’t the money supposed to go toward that?
This whole article and workaround exist because the Valheim developers have a problem that needs to be fixed. The answer is definitely not spinoff mod projects to take care of this issue. That would definitely be the worst timeline where the Valheim developers absolutely refuse to do this and have officially announced it will never be done ever. Even the workaround in this is just a band-aid. They need to change the multiplayer model sooner or later and they seem to be aware of that. I really recommend any effort go toward encouraging the Valheim developers to just either add some options to do this or adjust their multiplayer model.
The forked Minecraft server community seems really similar to this idea. I definitely have seen communities and ecosystems like you’re saying but I just don’t think we’re there yet with Valheim. It is not unreasonable to ask them to eliminate the bottlenecks/lag that is happening related to other clients controlling the monsters. They’ve made a bunch of changes to the send/receive limits and related systems before. They know it’s an issue. If we continue to let them know that the fixes they’ve done have not been enough to take care of this I’m sure they will make further changes related to this system.
This is a fundamental flaw in the design of the game that needs to be addressed. There’s still a lot of bosses left before the game is complete and time to take care of this. Don’t let the developers off the hook on this issue because if they actually fix it the right way it will be 100x better than this mod ever was. A modded .dll distribution is still a terrible solution compared to the developers fixing the multiplayer model. That would be an actual game changer. This workaround is a band-aid in comparison if that makes sense. Hopefully that helps explain it!
I’m going to try to see if this fixes my problems. If it does, I’ll upload the .dll for you to http://sketaful.se/valheim/dll-fix/
That page will just list whats in it and I’ll upload every new one I make when I make it, so it might be a day or two late sometimes.
What James is talking about with the “effort” of compiling etc… Well, as long as it’s already being complied since I need it for my own server, it’s really just an two-click upload to share it. 😉
Hey Sketaful,
It’s almost no effort to compile it. Almost anyone who has followed the guide once before can make the mod in 60 seconds.
That’s why I just give out the instructions. It’s *maintaining* it that is hard. 1000x times harder than making the mod.
In fact it’s so easy that it’s honestly never been an issue before. There’s only a handful of people that legitimately do not have the technical skills to follow how to do this guide that I’ve seen post here in the comments before. Even in those cases though every group historically has had at least one person competent enough to follow a guide with screenshots. Just have that person make the mod for your group and send it out. That’s what everyone else has done.
Not everyone needs to be a technical expert so I’m not even shaming anyone who this is over their head. It’s not so hard though that one of the more tech-savvy people in your group shouldn’t be able to handle it. If they can’t follow the guide then I don’t understand how they can follow enough of this technical discussion to even understand what the issue is. They probably aren’t in which case replacing files willy nilly is probably not a great idea.
I have no issue linking to this whatsoever. If you defy the odds and maintain this I will only congratulate and thank you. I wonder though if it will be even a month before people are posting this link/file is outdated but time will tell! How long does it take to completely beat the game with a group? It has only ever taken me a couple of weeks. Do you want to keep doing this when you aren’t playing anymore? How long will that be?
You’re actually the first person who has tried. I don’t have another example I can link to of someone trying this. I’d imagine if it happened very often I’d have a graveyard of examples to post to but I guess it’s the Mistlands release that has pushed the popularity back up where people think they want to temporarily take on these projects (for the moment). There was nearly a year of silence on this post until Mistlands was released and in a month or two I imagine it will be silence again until the next major boss release.
That’s why it’s difficult for me to buy into these ideas. I literally didn’t hear a word from anyone about this mod or this game for a year. There wasn’t enough interest to generate even 1 comment about it. Now when I see the big surge from Mistlands interest it’s hard for me to forget that year of silence right before this and think everyone is just back and is going to actively play Valheim forever now! No. I’m expecting exactly the same thing as before. This interest will die off within a month or two and everyone will be gone until the next update.
I see the same thing in the Minecraft server community. Everyone wants to run and start a project while they’re actively playing. There’s literally thousands and thousands and thousands of Minecraft projects. I would say 99% of them are abandoned. Literally. Why? Because it’s really *hard* to maintain something when you aren’t actively playing anymore. It takes someone truly special to do that. And by special I mean that those of us who do it probably have some sort of mental health disorder to explain why we’re so weird (a little bit of OCD in my case, the same reason I like mowing the lawn or plowing the driveway when most people don’t, it’s stress-relieving and super satisfying to me to see everything go back to perfect order as it should be and maintaining projects can sometimes give this satisfaction in the same way as monotonous tasks like mowing the lawn).
These projects surviving the gap is unthinkable to me but I have a dozen or so projects on GitHub I’ve released for years so my perspective on how hard it is to launch and maintain these projects is probably different than most people. It really is maintaining it that is the hard part. I came up with this entire variation of the mod in a fraction of the time it would take to maintain a project like you’re trying to do where someone has to build and distribute Valheim’s files for every. single. release.
It’s actually horrifying to me that this route seems easier than convincing the Valheim devs to fix their own game. I can assure you it’s not easier and that it almost certainly won’t work over the long term unless you are truly one of those special ones that can maintain something they aren’t playing anymore. I wish I could somehow gather and combine all of the effort out there going into things like this and just channel all of that power into spamming the Valheim devs that they better not dare add a single new additional feature before they fix this issue. This is why this project wouldn’t be satisfying to me in the way a lot of my other projects are. I would just be angry every single time I updated it at the devs for not wanting to fix their own game (and a little at the players too for not straight up demanding it).
I congratulate you on your massive balls. I literally do not have the balls to commit to this project of maintaining this because I know what that commitment really means. I shudder thinking every Valheim update of all the comments I would get saying “UPLOAD THE UPDATED .DLL ALREADY”. I’m already having nightmares about it even though I never agreed to do it and am not at all involved! You are even hosting this on a Swedish domain and Coffee Stain Studios is a Swedish company. That is bold. Either that or you know they won’t / can’t do anything to you for some reason (as an American I can guarantee if they filed a lawsuit against me for doing the same thing I’d get crushed). Best of luck, truly, I am legitimately rooting for you despite my skepticism!
Otherwise I’ll be seeing all of you again just like I saw many of the same people from launch and Hearth and Home returning (like myself) to play the new content only to find the Valheim developers have STILL not fixed this issue and it’s exactly the same as when I published this. I hope that’s not the case next time. I really hope we’re not back next boss sharing our frustration with each other this still was never patched/fixed.
The files will be maintained atleast as long as I have my own server up since it’s basically just uploading what I already do every time it’s needed. It might take me like a day extra sometimes if I’m working, but beside that it will be there.
Hey Sketaful,
That’s pretty much how I maintain my projects that are “downstream” from Minecraft. I’ll update the projects / containers within a day or so of a new release.
Best of luck, I’m sure a lot of people have downloaded it already! I’m definitely seeing a lot less problems from people doing the modification so I’m assuming a lot of people are just downloading your release. Keep at it (as long as you stay cool with Coffee Stain Studios that is)!
When it come to the legal aspect of uploading the files. Yes, it’s the property of Coffee Stain Studios but it’s not the actual game I’m uploading so either they care enough to tell me to pull it down or they don’t. I’m not hurting their business by hosting the files, I’d say I rather do the oposite. And Sweden. Lovely country. The effect a negative response from them would do is that the files will be removed (by me if they ask me to), they won’t press charges, and if, for some strange reason they will, they won’t win since they can’t prove it hurt them in any way.
Btw. I live like 7km (around 4 miles) from their studio so I guess I just head over there and ask them why if they do. 😛
Hey Sketaful,
That makes sense, I’d love to visit Sweden someday! I’d imagine if you hear from them it would be because of what I was alluding to in my earlier posts (they can lose certain legal protections if they ignore things like this).
I’d love to think that instead of try any legal action they would just decide to add this in as an option in the “Options” menu (whether in the server, the client, or both). That would really be the best thing for everyone I think (no matter where they live or what connection they have). That way people can set this to whatever makes sense for the connections they and their players are using!
Yeah, I wouldn’t hold my breath. They’ve made an amazing game but it’s not very well optimized.
I wonder, Does it help to do this not only on the dedicated server but on the game itself?
…and is there any difference between the game’s assembly_valheim.dll and the one on a dedicated server or is it enough to use it in both?
Also. While at it, changing false to true in a couple of places let you use the odins dlc ;P
Hey Sketaful,
Welcome back! So yes, there is a client version of this that can help if you are interested in offering it.
This is the client mod here with instructions from patk.
The client side mod is not as beneficial as the server-side mod but it has some really specific benefits. For example there are boss fights like the swamp boss fight that have a huge amount of entities on the screen. Doing the client side mod has significantly helped with scenarios like that because in those cases since we now know that the clients control the monsters we know that if there’s a lot of monsters that it has the potential to lag things simply because of the client’s send/receive limits instead of the server.
In other words you’re exactly right. This same mod can help on the client side. It’s not as helpful as the server one because the server is usually the one that is going to hit the send/receive limits. There are exceptions though such as major boss fights where since the clients control the monsters in those cases it can actually be the *client* lagging everything up. Those are the situations where the client side mod can help.
In fact for patk and a few other groups the client side mod was the *key* thing they needed to beat the game as a group. The server side one wasn’t enough because it was the boss fights that were really hurting them and it was due to the *clients* lagging. That’s why the client-side mod made a difference in those cases. It won’t have as general and noticeable of an impact but the situations where it does have an impact it can make all of the difference.
It would be distributed basically the same way. They just put this in the client .dll folder instead of the server .dll folder. Hopefully that helps, and thanks for continuing to maintain the binaries of the mod, people are loving it!
You will have to keep a view on the date of upload to figure out if it’s new enough.
Hello everyone! Just wanted to add our experience if anyone else oversees this problem when it’s not working to modify the file if the serverhost has a linux system and sent the .dll file to a windows user to quickly modify it and send it back.
We tried to just send the “assembly_valheim.dll” file over and change the values before sending it back and overwrite the old file in the servers folder. This made alot of errors popping up in dnSPY when you tried to compile the file after the numbers were edited. Turns out you need to zip/rar the whole “Managed” folder with all the .dll files included and send it over for it to work as it seems to be looking for other dlls mentioned in the code when you compile it. This might be obvious to an experienced user but we totally overlooked it before we finally found out why it didn’t work. It did work like a charm after we had the whole folder on the computer editing the file though, afterwards we just zipped the whole folder down again after changing it and sent it back to the linux server system for replacment.
And thanks for guide James! Super helpful and easy to follow, cheers!
~Charlie
Hey Charlie,
Great comment, thanks for leaving this! This makes sense. It sounds like what is happening is that dnSpy is resolving all the dependencies. That’s why you transferring the whole folder helped as now it’s resolving those correctly.
I definitely recommend anyone modifying Linux do the same. Transfer the whole folder. This should resolve the dependency errors. Thanks for sharing!
It’s the same for windows as well. It needs the whole folder to get it all correct. It looks for dendency files and if it doesn’t find them where it expects them it’s a ton of errors.
Hey James, I’ve got another question. I was reading through an earlier thread you were responding back and forth in about how the netcode is really P2P, and a lot of the issues stem from the fact that tons of data regarding monster behavior and such have to be sent back and forth between the player that is “controlling” the monsters with their client, the server, and the other players. This got me wondering:
Should this data rate change you document in this article be made to the client application for all the clients playing, in addition to the dedicated server?
Say my client is the one in charge of the monsters and such and I have 5 friends in a dungeon with me. Isn’t my data rate cap on my client going to get hit and cause lag issues? Wouldn’t it help to raise that cap on my client, and not just the dedicated server?
Please let me know if I’m missing some key piece of information here, such as the data rate being set separately in code for the client than for the server, for example
Hey orbtl,
Welcome back and great question! Some people say that it is like Greg thought that it was worth it here.
Also check out patk’s post here. Most people seem to say there is still a benefit to doing this but that it’s not as drastic as when you do it to the dedicated server. It’s exactly how you say (patk’s post goes into this as well). If the client controlling the monsters is choking on the send/receive limit then that can absolutely lag the entire server and raising the send/receive limit can help with that.
In reality it’s a lot less likely to happen to the client controlling the monsters than the server but these are two separate things. The server has a huge benefit to raising it and the clients can have a marginal (and situational) benefit from raising it if controlling the monsters is causing that client to hit their send/receive limits. You can do one, the other, or both fortunately so it’s not one or the other and this is totally possible and can help.
An example of a situation where modifying the clients has been documented to help is the swamp boss battle. In this fight the swamp boss will constantly spawn a ton of minions and it can quickly add up to dozens (and they’ll be combined with all the other swamp creatures in the area which there will usually be a bunch running around interfering with the fight). This is a *LOT* more monsters for someone to control than a normal situation (or just about any other situation in Valheim).
We’ve seen the client side modification make the difference in that fight before (combined with the server modification). This situation makes sense too because it’s the situation where you will have the most monsters naturally in a fight together and it’s going to bump up against the send/receive limits of the client controlling them (when in normal situations this is not likely).
It’s a lot more complicated to get all of the players connecting to all do the mod or distribute the file but people have done this before for their group. It’s a lot more work but there’s a few people in the past who did have further improvements by modifying the client dlls (exactly as your suggesting). It’s just a question of whether it’s worth the additional effort and complexity as you will have to make the new file and distribute it to everyone each update. It has been done before and people do get tired of doing it over the longer term every single patch but it absolutely can have benefits according to others who have tried it.
Hopefully that helps!
Okay thanks, that makes a lot of sense. I’m currently experiencing a lot of lag issues even with the increased limits and it seems like something else is limiting the data rate, and I’m not sure what. Was hoping maybe you know. I increased the values to 10x instead of the 3x recommended here, and I noticed in our base with a few players connect I’m capped out at data sent: 150kb/s. I have symmetric gigabit internet so I can handle way more than that, so I tried upping the number to 100x and it still won’t send more than 150kb/s. Do you know if there is some other variable somewhere that is limiting my data sent so that my client (when the chunk master) can send more data to handle all of the physics going on?
Hey orbtl,
That is basically patk and Greg’s journey from page 1 of the comments. Read patk’s thread here:
Thanks for this guide, once we implemented this guide with a lot of mods installed our send and receive caps out at 150kb/s and when this happens we get massive desync.The valuable content in this article is actually the comments and not the guide. The guide is just the easiest and most fruitful mod to do. It’s just a taste basically! If you want to go deep on this though yes you can go much much further and we already have/did back then pretty thoroughly including lifting the 150kb/sec cap.
If you go back from comment page 1 and just read through them you will know everything you need to know for sure. It’s from a year ago and that will save you a lot of time as you’re kind of repeating patk/Greg’s journeys and they documented the journey very well (making it easy to repeat). A bunch of others have chimed in as well and this has all been thoroughly investigated/tested on the 150kb/sec limit you’re seeing (which can be changed, yes). In other words they already took the hits for you on this and left a record of it so you don’t have to start from square one fortunately.
There are a bunch of pitfalls to what you are trying to do. For example if everyone’s send/receive limits don’t match there is massive packet loss. Definitely read through that whole thread though from patk as he did this and outlines exactly what to do for the 150kb/sec problem and some of the pitfalls to watch out for. If it’s all done right people have had successes for sure.
The short answer is that yes there is more to the send/receive limits than what the guide is modifying. There’s additional code in there in those comments on how to do it but if you do it everyone’s needs to match that is playing or it will make things very very strange. It’s really hard to describe it better than that though as it’s all written down on page 1 of the comments and will describe it better than I am here. Hopefully that helps!
This doesn’t seem to work for my Linux Dedicated server. The server crashes shortly after starting. Do you have any hints on how to properly edit this file for a Linux Dedicated server ?
Hey Russ,
Everyone who has had trouble with this in the past was trying to edit the Windows version of the file and use it on Linux. Even though the game is in .NET these files are actually very different from each other and not compatible. The guide will basically work exactly the same but you will need to physically retrieve the Linux version of the files from the machine, edit it, and then put them back.
There was this case here.
And another case here.
As well as this case here.
In all cases they were not modifying the right files. If you are trying to modify the Linux files from a Windows machine (which is what literally everyone else who has tried this was doing that I can recall) then you actually need to get the files from the Linux machine and transfer them to your Windows machine and then make the modifications.
The files are different between the different platforms. The crashes tell you for sure you are using one for the wrong platform/architecture. The modifications we are making will not crash it even if you turn the number way, way higher than you should (it will just swamp your connection). The crashes basically mean that the wrong files / platform / version is being used somehow (almost certainly the Windows version).
I’d definitely try pulling the right files from the Linux machine directly and editing them. Make sure it’s not using dnSpy’s cache and is actually opening the file that you think it is. We just had an issue yesterday where dnSpy was caching older versions of the file despite them being positive they opened the newer one (and I believe them). A lot of reverse engineering tools work this way and have a cache/database and it’s not always obvious that is how it’s working and in fact can sometimes be quite difficult to refresh unless you’re pretty experienced with the products. It’s very quirky and you will probably need to completely close all existing edits and files out. Double and triple check this because a crash indicates the wrong files are being edited for sure.
It’s fine to modify them using Windows but the process is you need to get the file from the Linux machine, edit it on Windows and build the new version then take it and put it back on the Linux machine. This has been done before a few times but it’s extremely important to use the Linux files as the Windows ones will definitely lead to this exact crash. .NET apparently doesn’t work like Java where you can run the same .jar on every platform. The .dll files are actually different between the two platforms (although the code inside dnSpy will look identical, it’s probably only the dependencies/linking/etc. that is different).
Hopefully that helps!
Thanks for the info. I pulled the files directly from the Linux server, as it is also in my office. But I initially had the Windows files open from my own installation on my PC, where I was editing, so I think I was a victim of the cached files. I appreciate your insights. Thanks for the guide!
Hey Russ,
No problem at all, I suspect it’s the cached files issue as well. Shockingly orbtl’s dnSpy copy managed to pull up a cached version from launch after all of this time. I’m sure those files are long gone and that they must have only existed in dnSpy’s cache at this point (which is probably why they understandably thought this was a recent copy of the current dll on the system).
Thanks for letting us know what it was and ended up looking like. This will help others as well especially working with the Linux files. Enjoy and take care!
Hello! Thanks for this wonderful guide. I remember following these modifications over a year ago and they were very helpful. However I have just returned to the game and it seems like the code has changed. Do you know if there’s a way to still make these changes with the changes to the netcode (or if they are still necessary?). The SendZDOs method that contained the code you modified appears to have been rewritten and no longer contains that hard-coded integer value you were modifying. Here is what I see in the code now:
private int SendZDOs(ZDOMan.ZDOPeer peer, bool flush){
if (!flush && peer.m_peer.m_socket.IsSending())
{
return 0;
}
float time = Time.time;
this.m_tempToSync.Clear();
this.CreateSyncList(peer, this.m_tempToSync);
if (this.m_tempToSync.Count <= 0)
{
return 0;
}
int num = this.m_dataPerSec / 20;
ZPackage zpackage = new ZPackage();
ZPackage zpackage2 = new ZPackage();
int num2 = 0;
for (int i = 0; i num)
{
break;
}
}
}
if (num2 > 0)
{
zpackage.Write(ZDOID.None);
peer.m_peer.m_rpc.Invoke("ZDOData", new object[]
{
zpackage
});
}
return num2;
}
Hey orbtl,
That’s very interesting. It actually looks more like the old method.
Can you try following my old guide and see if you can find the m_dataPerSec variable? This is very strange though that they would completely change it back to the old method. It’s back to m_dataPerSec according to your code. This entire article was written because that variable was completely removed from Valheim at the time but my original guide for this style of code is still up. This was how it worked at launch actually.
Unless this is a copy installed from hard media that hasn’t been updated (did they even ship this game on hard media?) or I can’t even imagine how this could happen unless they actually have changed it back in the current version (unless you restored files from a really old backup over the top of a newer install or something).
Are you perhaps editing the client .dll instead of the dedicated server .dll? I’m not sure they ever changed the client away from m_dataPerSec. Modifying each client can have marginal benefits but testing from back in the day showed that 90% of the benefit seems to be boosting the send/receive limits on the dedicated server (which is acting as a relay for everyone).
It makes sense that pretty much all of the benefits of raising the send/receive limits will be on the server since the send/receive limit can basically be thought of in bytes per second. The server is relaying for *all* of the clients so it’s going to bump up against the ceiling way before any clients will simply due to the fact that for any of the clients to talk to each other (which they do in Valheim even going as far as controlling each other’s monsters) it has to be relayed through the dedicated server. It has a lot more to send/receive as the server than any client ever will which is why it hits the send/receive limits first.
Either way you should be covered by following the old guide instead!
Thank you so much. I guess dnSpy keeps a cache of files you have opened previously or something? Once you pointed out that this was actually the old code, it made me realize this must be the case. I made sure to hard close everything in dnSpy and re-open the .dll again and the new code showed up as it does in this article. Sorry for my mistake and thanks so much for quickly helping me figure out what was wrong!
Hey orbtl,
No worries at all, that actually makes sense that it could be something like that! It’s not uncommon for other reverse engineering tools (Ghidra / IDA Pro come to mind) to similarly cache/import a dll/exe/etc. It would almost be hard to find some of these files if you were looking for them intentionally as that has been gone from the server for quite a while.
I’m glad that helped and thanks for leaving the comment! Surely there are others that will be coming back to try Mistlands that will probably have the same thing happen and hopefully this saves them some time.
Enjoy and take care!
Hello, im running a dedicated server and i have some of those sync/lag issues. We are 4-6 people playing constantly.
Sometimes doors and chests take a long time to open. The ennemies/animals look like they are teleporting.
Do you think its a CPU/RAM issue or more like the problem with server’s send/receive limit ?
Is this post still valid in november 2022 ?
Thank you!
Hey Patrex,
Yes, that definitely sounds like the normal issue. The amount of players is still about right as well. I’ve started playing the Mistlands as well with my wife but I haven’t been encountering the lag with only 2 people on.
My understanding from talking to others who have done the fix is that this does still actually help a little bit. The Valheim team claims they raised the send/receive limits but check out my conversation with Gesper here from only a month ago.
People have still noticed some benefits doing a process like this but the problems are mainly due to scaling with Valheim. Modifying the send/receive limits can help reduce/alleviate symptoms, yes, but it can’t eliminate the problem. The amount of stress on the server going from 4 to 6 is quite substantial. It definitely doesn’t increase the load in a linear way to add players. It’s exponential because it has a heavy peer-to-peer component.
That also means the lag can come from other players. It doesn’t even necessarily need to be that someone has a bad connection to the server. The players having a bad connection to *each other* will also lag the server. This is why it’s really complicated and why this is only a tool that can help and not really a solution. The solution is Valheim would need more scalable network code (probably a rewrite on the network code).
I’d say the state of this issue and the potential fixes are exactly the same as they’ve always been. The send/receive limits are one tool you can use to try to reduce the lag caused by Valheim’s scaling issues. It won’t fix them now and if you read the older comments from back in the day it didn’t fix it back then either. Did it make things marginally better though? Yes, and jacking the send/receive limits up higher today can help sometimes for the same reason it did back then.
It depends on exactly what the issue is causing the lag on your server. This issue is absolutely identical to launch in my opinion. Nothing has changed. Valheim’s scaling issues will get you into trouble with the send/receive limits really quick today on any server with 4-6 people as it will overwhelm your client’s networking capabilities even if you are running it with a dedicated server in a data center.
A lot of the people who commented on this originally had enterprise equipment. It doesn’t matter. The lag isn’t necessarily coming from the server but from the process of all the clients exchanging P2P data with each other (and things in the game basically stop until they’ve all spoken and come to an agreement about what is going to happen on everyone’s screen, it’s a silly system).
An example of a successful “fix” from this would be people being able to do a boss fight without lagging so bad they couldn’t do anything. Some of the biggest success stories using this method are exactly that. They nibbled around the edges enough so they could get through one fight without being miserable. In some cases that was honestly enough to make people and the group very happy as it was that one boss fight that was holding them back from beating the entire game as a group.
It was simply a matter of nibbling at the edges of the networking stack to try and make up for the multiplayer design of the game which you can still do today. It will never be able to support more than the amount of players you are saying though without a networking code rewrite. It only gets worse if you went to 7 or 8. Much worse.
Understand though that there was never what people would call a true “fix” where you fix some setting and never have to worry about this again. That’s not possible without a network code overhaul and essentially a completely different system that drops all the P2P stuff. It’s more like a mod to try and tilt the scales in a slightly unplayable situation to playable by basically manipulating the network stack to use inappropriately large send/receive buffer sizes that would be completely unreasonable in any other situation because it means they chose a poor design for their networking model. In retrospect though I should have never used the word “fixing” in the title as the only people that can truly “fix” this are the Valheim developers.
Normally as a developer this is the part where you realize you need to change your networking model because your original choice won’t scale and making the send/receive buffers enormous is a hack that won’t fix anything and only buys you some more breathing room (in the form of maybe having 1 or 2 additional people online) before the problems hit.
This type of networking is what things like the original Xbox Live or older Playstations used. This is why most of the older consoles only supported a max of 8 players (and sometimes not even that). They used exactly this type of networking and a lot of those console games would lag with the same scaling issues as Valheim if you actually got 8 people in the lobby (apart from the nightmare of the host logging off and having to migrate to a new host and a bunch of other fun P2P stuff if people remember that).
I would venture to guess whoever wrote the multiplayer code from Valheim must come from a game background that used this type of networking and I don’t understand why you would choose a P2P model when you literally have/offer a “dedicated server”. In Valheim the dedicated server really just means it stores the map file. It’s still a P2P game even with a “dedicated server” which yes makes no sense but that is what it is.
Although Valheim’s developers say they’ve raised the send/receive limit in the code all that means is they implemented this same hack that I’ve wrote about in this article into their code instead of actually fixing the underlying problem. Instead of overhauling the networking code like I’ve always said is what needs to happen instead they implemented my hack into their code and called it good which has led us to today where sure enough this still isn’t fixed.
You can still turn the setting higher than them though. In the end this is a number in code and you have to choose how big it is. My change I wrote about here just makes that number bigger than they did. You can still make it bigger than they did today if that makes sense. It does mean you’ll probably get diminishing returns out of raising it though. They already raised it at least once but again you can always raise it higher.
You get to a point though where your send/receive limits are so high you will hit the limits of your player’s connections. It’s game over at that point. There’s nothing else to milk or tap. You have to rewrite the game’s multiplayer code at that point as it’s absurd that these players can likely stream 4K video with ease but can’t play Valheim with 6 players. You can watch 4K video don’t have enough bandwidth to play Valheim with 6 players using this model which is kind of hilarious to think about and shows how bad the scaling really is (and how much data we are talking about that is getting duplicated and sent between all these different P2P clients). That’s a problem with the game itself. A structural problem (just to explain).
Maybe a second wave of popularity bringing light to this issue will prompt them to finally rewrite the multiplayer networking code? That’s what needs to happen though and hopefully my super long-winded explanation provides some light on why this happens at least.
Hopefully that helps!
Thanks for the detailed reply.
Yesterday we were 9 on the server and i noticed that the problem is only when we are in the same sector of the map.
If i take a portal to another continent far away, i dont have any sync issues.
But as soon as i come back to our village, i can see the problem (Mobs teleporting, delay on doors, chest etc)
Only with 3-4 person in the village and its very intense, cant even fight a group of Greydwarves
Its kinda sad since the point of multiplayer is to play in group with friends…
I also found a post from one of the Devs who said the game is not using a P2P network :
I’ll still try your fix tomorrow and we will see if its a little better at least, but i hope they fix the code soon…
Hey Patrex,
That developer is playing games with the definition:
Thanks Richard. I understand the nuance but it was easier to say P2P than explain the nuance. I fully appreciate that it's complicated and appreciate the response.I think there are two classes of users,
those with DSL who are limited by the transmit rate. It's very close to DSL limits
those with lossy networks.
The former can really only be addressed by limiting the bandwidth. short of some compression scheme the one obvious approach would be to halve the update rate. Makes for some choppy behavior but it would work. There are some other ideas but get much more complicated, e.g. balance responsibility for objects by network capacity but that risks clobbering machines with great internet and poor processing.
The latter can be addressed in small part by posting a sticky here with instructions for how to diagnose and correct most forms of loss. I've seen that done in other games and would be happy to help contribute. It has the added advantage of helping users QoL beyond the game. After that there's not a lot one can do for lossy traffic with this server model.
Don’t be fooled by this. The developer is saying technically it’s not a P2P game and that it’s more “nuanced”. Do you know what that nuance is? It’s what I spoke about in my previous post. It works like a P2P game even if it technically isn’t one. It might as well be P2P since it has the same restrictions and limitations. Yes, we know it’s technically not a P2P game, it has a dedicated server. That dedicated server is fake though (or at the very least calling it a dedicated server is misleading). It doesn’t work like the dedicated server in other games and is a glorified map storage computer.
On other games the server is controlling literally everything (in most games actually) and it generally does not care what the clients are or aren’t saying (nor does it trust it for security reasons). Obviously there are limits to this like if someone completely lags out they are going to just stand there on the server until they drop. The whole server doesn’t stop though and wait for clients to report what they’re seeing on their screen like it does in Valheim (nor should it, this is how people make very nasty exploits). People’s game clients are still running the show in Valheim even with a dedicated server and it’s such a strange and inappropriate meshing of these two types of multiplayer models.
Look what the developer said here:
It is however pretty sensitive to one player having a bad connection. If player-A is controlling the monsters in the area has high ping to the server then player-B will also notice thisThis is what I’ve been trying to explain. The Valheim clients are controlling the monsters! You read that correctly! No, this is not how other multiplayer games work. They said this lets them do a lot more things like physics client side. As someone who has developed multiplayer games before and have released several of them on my GitHub I cannot believe they are having the CLIENTS control other people’s monsters in MULTIPLAYER! WHAT?! This is exactly why it lags and is essentially a P2P game even though they are quibbling with the definition. He just said it. Player A’s client is controlling player B’s monsters! In multiplayer! When they have a dedicated server! Why?!
If we are going to talk about nuance though they have changed some things to not rely on the clients so heavily. For example now in Valheim when the server is lagging you may see people moving now. That didn’t used to happen. It’s because they moved that and a couple of other things out of the main single-cored thread that does all of the game logic. Watch for this when you are playing. You may see people move around and not be able to do anything else.
That’s because 99% of everything in the game is still single-cored and all of that will stop. Only movement and I think like maybe opening doors or something got moved outside the main single-cored game logic thread? It wasn’t much though. This only makes it look like the server is still operating because people are technically moving around but you won’t be able to use any abilities or do anything if that makes sense. This is just a visual trick though that doesn’t solve the problem. It just makes it look less jarring when people keep moving generally in the direction they were going instead of just stop dead in their tracks but the server is in fact still stopped dead in it’s tracks for all of the other functions that are not broken off the main game logic thread.
This is so easy to disprove. How many of you have DSL? I bet you probably have something greater than DSL and are still experiencing this issue. Can you play other multiplayer games? Have you ever experienced anything like you have in Valheim before on DSL or otherwise? No? Other games don’t work this way? Exactly. Valheim definitely is the issue for sure here!
Look at the sentence at the very end:
there's not a lot one can do for lossy traffic with this server modelBOOM. There it is. This person responding and I completely agree and reached the same conclusion. There’s not a lot one can do with this server model. That is my *exact* argument. The Valheim server needs a different multiplayer model to solve this problem!
Honestly my takeaway from that developer post is that they knew all the way back then that this needs a new multiplayer model to solve this problem. I think I’m more annoyed after reading it and was more content believing that they just didn’t really understand the problem or something like that. They actually explained some things I didn’t dare to say directly such as Player A is controlling Player B’s monsters.
To be fair to the developers the approach they are using has it’s advantages. You know how they mentioned physics? A common problem with dedicated servers in other games is your hits not registering. Valheim is pretty forgiving with hit registration (although your hits may take a long time to appear if the whole thing is all lagged up, they *will* appear eventually). Your hits will always register because they’re cheating. They aren’t calculating the hits on the server. They are literally taking your client’s word for it and the decisions are made based on what your client says. I couldn’t disagree with their decision more to use a model like this though as the downsides far, far, far outweigh slightly better hit registration (which shouldn’t be an issue anyway on a well configured dedicated server).
I knew it worked something like that but there it is straight from the developers. This is why it’s a P2P game no matter what they say technically qualifies it as not P2P. If other players are controlling other people’s monsters (and not the server) that is P2P. Period. And with all the disadvantages that come with it such as one player being able to lag an entire dedicated server (which should be impossible).
It really doesn’t make it any less P2P that they wrap all of this through a dedicated server. It seems like they mean that it doesn’t work like a torrent does where you have a bunch of connections with each other. No, this system is actually worse than that. It has to all pass through the dedicated server who would serve as a relay (or the glorified map storage machine I alluded to earlier) sending these back and forth between clients instead of them talking directly.
That’s actually less efficient than P2P in a lot of cases. It’s P2P but with the additional step of everything having to be relayed through the dedicated server instead of sent directly from one peer to another. At the very least it doubles everyone’s ping effectively and that’s just from the math of having to go to the server, then go to the player, then go back to the server and then back to the original player again. In P2P it would look something like packet -> response. In this model it’s packet -> server -> player destination -> response sent back to server -> response sent from server back to original client. That’s a lot more steps for even a ping / pong.
It would be better if they picked one or the other. Go P2P or go dedicated server. At least then we would get the advantages of either P2P or a dedicated server instead of the disadvantages of both.
I honestly think now would be the time for the Valheim developers to do this. They just got Mistlands out and from what I’ve seen so far there’s quite a bit to do. There’s still several more bosses/biomes to go before the game is completed. They can be absolutely forgiven for choosing the wrong multiplayer model initially as they were a very small team at launch. I hope they don’t wait for any more bosses to address this issue. Hopefully that clarifies a bit more of what is happening here!
Hello James and Patrex,
A small status update from Mistlands:
I upgraded our server to Mistlands too. We are now up to 6 players and tried playing without this fix at first within the new update.
We encountered still the same problem, nothing has changed with regard to the lag problems.
My hope was that when Mistlands drops for the console release, the main focus of the developers will be on the lagfix and that was also the reason for the long waiting time. But I was wrong. I admit I am a little disappointed what they have been doing all this time aside from implementing a few new models (/no hate).
Anyway, I applied all changes from this fix again on my server assembly_valheim.dll and got the same results as before the update. A lot more playable/enjoyable especially when player 4 and 5 hits the same area and I would recommand this fix to everyone -> Big thanks to James. I repeat: The lag problem is still! exactly like James figured it out on comment page 3 in our discussion.
I also recommand for all server hosts to ensure that every client has done its FPS boost (boot.ini configuration, disabled steam overlay, plays with “-window-mode exclusive -screen-fullscreen” and sets Task Manager priority to high). In case there are players on the server who are not playing on a high end device, it’s better to avoid extreme FPS drops.
Remember: The lag is as bad as the weakest link for all players in one area.
Best regards and keep up the good work!
Hey Gesper,
Thank you so much for verifying this! I had a bigger group I was playing with the last time but this time it’s just my wife and I playing at the moment so I didn’t have enough players to test this.
We know for sure that they haven’t changed the multiplayer model though so I knew that for all the technical reasons that we’ve covered this issue shouldn’t have changed (or the potential solutions).
I wasn’t sure what kind of diminishing returns you’d see as the Valheim team had also raised their own limit but as I’d outlined before at the end of the day they have to pick a number. They are never going to pick as high of a number as you will because too high of a number will completely overwhelm people’s connections.
They’ll never be able to safely set it for the entire world as high as you could set it yourself (with the caveat that you may swamp your own or your player’s connections going too crazy with it). They have to take into account computers and connections that are far less capable than anything you or your group are likely to be willing to use when it comes to setting a worldwide global release value.
It’s just honestly too bad they still haven’t made this a configurable option. You have to directly modify the .dll files still to this day to change this unfortunately. On many other game servers (especially Steam which Valheim has integration with even) you can actually just set the send/receive buffers value right in a config.txt file or something like that depending on the game. In Valheim this is still a hard-coded value.
Situations like this though are exactly why many other game servers let you configure this. You can’t ship a game with a perfect preset value for the whole world when it comes to networking or even hardware (which is why there are so many options). It’s almost always going to end up conservative/too low for the vast majority of likely players in order to make the game playable on the really poor connections/computers out there.
I’m very grateful to you for testing and reporting back that this workaround is still helping and showing noticeable impacts. Enjoy and take care!
It’s basically a P2P game with one server between that only have one job, to register all the changes made to the map when you’re offline. You could basically have it as a P2P game which keeps the latest mapchanges on OneDrive so everyone downloads that when starting their games. 😛
Hey Sketaful,
Yes, you’re exactly right and that’s a pretty funny analogy to think about this with. It absolutely might as well be a cloud hosted map. It wouldn’t impact the mechanics of the game at all since we know and have discussed at length the client-controlling aspects of the monsters and other parts of the game.
In fact if you go in and back up the server data there is literally only a map file to back up. All of the players profiles are hosted on their accounts / their computers. It is just the map file and the chest contents pretty much. Beyond that the server is only serving as a relay between the clients (less ideal than P2P even honestly since in P2P it’s a direct link vs. a middleman “dedicated” server)!