Performance tweaks for the game
Wed, 11/02/2011 - 03:55
#102
Does GMod give you hats?
Spiral-spy uses game resources in a creative manner unrelated to the game. AFAIK, there's nothing you can do in spiral-spy that will affect the game itself.
The stub launcher, when used as (assumed) intended by Eltia, is not much different from the .bat files discussed on the first page of this thread. However, both methods allow the possibility to operate the game in unintended, malicious ways. It's hard to tiptoe around the major security risk of OK-ing the use of unofficial launchers. Very Bad Things can happen if they are abused. Unfortunately, they can facilitate larger offenses of the ToS even if they aren't considered violations themselves.
Wed, 11/02/2011 - 06:11
#103
@Corsa
As explained here on why a stub was developed instead of using a batch compiler. It's not about the efficiency but more about the risk involved, especially given the technical level of the majority of the readers on this forum. The stub has this advantage that it has source code available and it is fairly simple (39 lines of C). So you can always look at the code before you compile and use it. (And that's one reason why I do not want to distribute a binary package. The tool should be used for experimentation and proof of concept. It is not a permanent solution that you can hand to people who have little to no programming experience.)
Next week I will be testing it on Shadow Lair. If this works, I am sure it will help save people a lot of CE (because people die less to lag). If you really, really want to distribute a binary package, please do not do it on this forum. Find another place outside of OOO's jurisdiction instead.
But of course, the fix is in OOO's hands. All they need to do is add 3 lines to their master getdown.txt file. My advice is that allow users to pick different performance profiles based on their preferences. Guys like me (and most people posted in this thread) have more powerful gaming rig and we should be allowed to use more CPU intensive profiles that give better gaming (lagfree) experience. People who run the game from browsers or legacy hardware, should probably stick with the current configuration.
In conclusion, this game is so much more enjoyable without the lag. I wish everyone can experience it in the future.
One last thing. any non-trivial Java programs will never be written once and run everywhere. OOO developers should not buy into this marketing BS.
Wed, 11/02/2011 - 11:17
#104
A couple notes here for those
A couple notes here for those poking around:
Depending on your computer Java is already running in server complier mode by default. Any 64-bit OS or a 32-bit OS with at least 2 CPU cores and 2GB of RAM. This is called Java Ergonomics. This also implies -XX:+UseParallelGC, Xms of 1/64th physical memory and Xmx of 1/4th physical memory. All of these settings can be manually overridden. In the case of Spiral Knights the Xmx parameter is forced.
A 256mb heap size is enough for almost all of the game if running minimum display settings (including compatibility mode), but ramp up the settings and hit a boss lair and you run into trouble. An aggressive garbage collector kicks in, pausing all other threads to clear up some memory. In some areas the game wants/needs more memory beyond the set cap and you can actually watch memory usage oscillate around the 512MB mark in the Task Manager.
Increasing the Xmx value is, by far, the single most effective way around this. This allows the VM greater elbow room and the parrallel GC to quietly work in an effective manner. On a server class machines I've observed it as more than effective enough, providing enough memory was allocated in the first place. Xmx384m seems good enough for most of the game play, though some further tests are pending. Xmx512m is moving into the more than enough territory, Xmx1024m is overkill and Xmx2048m is hilarious. The trouble with aggressively increasing the Xmx value is that you can blow minimum spec users out of the water with it (though watching javaw.exe eat through more memory than a modded Oblivion game was kinda funny).
Remember, specifying any values in the Steam launcher options or web version shortcut is useless since those options are sent (if at all) to the getdown VM, which launches a new VM for the main game. Firing up the game from the command of launcher.log works a treat though, but doesn't allow Steam login.
Java/SK does run multicore (SK usage on a Q9550), however it's mostly one super thread with smaller helpers that don't necessarily make a blip on usage statistics. The way Java Ergonomics is defining server class machines implies the parallel GC ideally runs on a separate core.
Wed, 11/02/2011 - 11:40
#105
SK uses multicores
Just that as the previous guy mentioned, it does not use the cores evenly. So when you have one major worker thread doing most of the stuff and a moderate master thread controlling the smaller drone threads, the result is that the overall performance is capped by how fast the major worker thread works. Hence, it becomes depending more on core speed than number of cores.
This is an acceptable design decision from OOO, as this game is developed before mutlicore CPUs have become mainstream and having many equally loaded worker threads won't do as good as one major worker thread with a few minor drone threads. This is because threads have bookkeeping overheads and they could disrupt cache coherence among the cores (=> more traffic on the system bus).
Wed, 11/02/2011 - 12:11
#106
Wed, 11/02/2011 - 12:31
#107
So...
I reread all the posts, and nada. I think part of the problem is I have no idea where to begin. I looked up how to create stub launchers for javaw.exe, and all I came up with were obscure websites and the link back to this thread. It would help if I had some inkling of how C++ works, but I haven't any clue. Can someone at least tell me what to look up? As it is, it would be nice if someone gave a step by step walkthrough, but I have always noticed that people who know how to program have very little patience with people who do not.
Wed, 11/02/2011 - 15:23
#108
@kirnan
Kirnan, I dont know how much more I can really go, theres enough hints and tips to get you to the source, if you search the correct things on google you will find it. next google how to compile c source code cygwin and all that it requires is a dll file and renaming one file. I hope im not getting myself in trouble for saying that much.
Wed, 11/02/2011 - 16:16
#109
Well
That definitely helps a bit. If anyone wants to further help or discuss this, we can always do so on our Steam pages....mine is lifesbrink.
Fri, 11/04/2011 - 04:59
#110
Steam Process Start Parameters
Couldn't find anything related to the little code posted, so I investigated a bit more how steam starts SK.
With System Explorer I could see the exact parameters:
javaw.exe -Dcheck_unpacked=true -jar getdown-pro.jar . client abc Stammdateiname="D:\Programme\Steam\steam.exe"
The last part is german (Stammdateiname="D:\Programme\Steam\steam.exe") and means the parent process filepath. This is automatically added when starting a new process. I don't know how to imitate starting javaw as steam.exe, since just running this line doesn't work, since the system automatically adds another (Stammdateiname=) part at the end of the parameters.
Edit: With "doesn't work" I mean it starts not as steam client. So no steam login.
So the solution would be maybe to set this paramter by ourself.
Fri, 11/04/2011 - 05:29
#111
@Knightmehr
Nice find. I didn't bother trying to see exactly what Steam's parameters were. The -Dcheck_unpacked=true parameter tells the launcher to unpack and verify the game's files since Steam will download the packed version (to save time and bandwidth). -jar getdown-pro.jar . is standard execution of the launcher. client is a keyword required by getdown-pro since, theoretically, the same getdown-pro can be used to run multiple programs. I don't have any insight on what the parameters after client do. They are passed to the game by the launcher, not to the game's JVM. So, unfortunately, parameters after client will not help in tweaking performance. Probably.
Edit: One more thing worth mentioning. Whatever Steam is telling the launcher and/or game to allow the use of Steam log-in, it isn't showing up in the launcher's log file.
Fri, 11/04/2011 - 06:30
#112
@Knightmehr
What I noticed was that javaw.exe got called twice. So the call to the actual SK instance that runs your game, isn't called by the Users. As a result, whatever parameters you pass to the first call, may or may not get passed to the actual SK instance. (Thus explaining why we got varying results earlier before the stub was introduced, because some specified parameters never reached the actual SK instance.)
If you follow what I said above, and you have some proficiency in programming, it's not difficult to develop your own stub.
Fri, 11/04/2011 - 10:15
#113
Gr.
I have no proficiency in programming...
Fri, 11/04/2011 - 14:27
#114
@Kirnan, Eltia
But me. But I'm having here problems compiling a complete jvm. Trying to compile openJDK my own java.exe, but I'm still at configuring the project. I think that's not the way you did it ...
So how to compile just a new java.exe/javaw.exe? I think I have the sources, but getting thousands of link fails, cause the rest of the jvm isn't build (openJDK). Also tried the jdk7 from oracle, there were also sources in it, but couldn't find the jvm.h there, that was the reason to try it with openJDK.
I would appreciate little help on this. Something I would discuss by PM but there is no such feature here.
Fri, 11/04/2011 - 16:19
#115
@Knightmehr
Well you do not recompile the whole JVM (that's overkill). All you need to do is, "trick" Steam to think it is launching the right JVM and "trick" SK to think it is launching itself properly, and you "sneak" the required parameters to JVM without them knowing.
Fri, 11/04/2011 - 18:01
#116
Hi guys !
any way to increase
Hi guys !
any way to increase performance on a mac running on osx Leopard ?
I constantly get stutter and lag on it, even running latest java...
I have 2 macs...one that I upgraded to Lion, and all problems disappeared ( i used to have exact same problem) and another imac that i cant upgrade to lion (because is a coreduo only )
I ve sent mail to support, but got nothing that helped...
cheers,
Sat, 11/05/2011 - 15:23
#117
@Eltia
I got it work. Thank you for changing my little spiral life.
Fri, 11/11/2011 - 21:06
#118
There already is a way to
There already is a way to configure the flags with which the JVM running Spiral Knights is launched: Any flags added to a file called "extra.txt" in the Spiral Knights directory will be added.
Now, first, a few warnings:
* This is not a supported way of modifying the game - if something breaks, whether it's the game or anything else that may be affected by a misconfiguration, we will not be able to offer support or take responsibility. You do this entirely at your own risk, if you choose to do so. Specifically, if you try to increase the heap space as described below and your OS/machine does not have sufficient memory, Spiral Knights will fail to launch. In that case, delete extra.txt again.
* This is also a method really only applicable for somewhat tech-savvy users. If you don't feel comfortable doing anything described below, just wait, as we will address the primary issue discussed here in an upcoming patch.
* You should only use an extra.txt that you either created yourself or at least looked at and understand fully. A maliscious person asking you to place an extra.txt in your Spiral Knights directory could do bad things in there.
With all that said, the much discussed increase of heap space is something that in our own testing did help in several cases. If you think you might be affected by this issue, here is how you can attempt to use this method for a fix until we address this ourselves (keep the above warnings in mind, though!):
Navigate to your Spiral Knights directory. This can be in several places, depending on your OS and method of installation. Typical examples are:
Windows Vista/7 Web Client - C:\Users\\AppData\LocalLow\spiral
Windows XP/2000 Web Client - C:\Documents and Settings\\Application Data\LocalLow\spiral
Windows Steam Client - C:\Program Files\Steam\steamapps\common\spiral knights
Windows 64-bit Steam Client - C:\Program Files (x86)\Steam\steamapps\common\spiral knights
Linux - /home//spiral/
Linux Game Prefs - /home/user/.java/.userPrefs/projectx/prefs.xml
Mac Web Client - ~/Library/Application Support/spiral/
Mac Steam Client - ~/Library/Application Support/Steam/SteamApps/common/spiral knights
These may differ if you chose a different directory at installation.
In this location, create a file called extra.txt (e.g. using Notepad in Windows, or any other text editor of your choice) containing the settings you want to use.
The flag to use if you want to try increasing heap space (for 512MB maximum heap space as an example) is this:
-Xmx512M
So, that extra.txt you create should just contain the above line.
Lastly, for the even more technically inclined: While you can (with the "own risk" restriction in mind) experiment with using other flags, be aware that Spiral Knights actually adjusts garbage collection settings itself at runtime, so any adjustments you do there might not be doing what you expect.
Fri, 11/11/2011 - 23:50
#119
@Endymion
Thank you very much! This is exactly what we wanted.
Thanks for the fair bit of warning. I think after 3 pages of tweaking, people are well aware of the risk (and hopefully no PC has been harmed in the process, yet).
So time to post our extra.txt file? I'll start:
-XX:+UseConcMarkSweepGC
-XX:+CMSIncrementalMode
-XX:+ExplicitGCInvokesConcurrent
-Xms512M
-Xmx1G
-Dsun.java2d.d3d=true
-Dsun.java2d.ddscale=true
-Dsun.java2d.translaccel=true
-Dsun.java2d.ddforcevram=true
Sat, 11/12/2011 - 00:14
#120
Thank You
Thanks for the info, wish this was addressed before the steps that have been taken were taken how ever im glad to know that we can create a extra.txt file and add the parameters. I my self will be testing it out, I've been loving the lag free game play with the stub now this is a good fix I can tell people who complain to come read and try if their comfortable. Thanks again.
Sat, 11/12/2011 - 00:20
#121
/cheer
Thanks Endymion! I wasn't expecting this announcement until next week but I'm very happy about it all the same.
This is my current extra.txt:
-server
-XX:+UseConcMarkSweepGC
-XX:+CMSIncrementalMode
-XX:+CMSIncrementalPacing
-XX:+ExplicitGCInvokesConcurrent
I'm playing on Windows 7 64-bit via Steam.
Sat, 11/12/2011 - 10:42
#122
Performance profiles
I have made three performance profiles (medium, very high and maximum). You can find them here. Feel free to comment on the profiles and I will make adjustment as needed.
Also if you are a Mac user and you have a profile you like to share, please post in this thread and I'll include it in the link above.
Sat, 11/12/2011 - 11:20
#123
I've been sucessfully using
I've been sucessfully using nothing more than an appended -Xmx512m to my launcher line for a while now, and it seems to be the only change I need. Default garbage collection behavior works fine with no pauses (full 64-bit system) and I don't use up to the limit of the heap space. I've tried Corsa's lines (without a memory change) and it just seems to lenghten the amount of time you can play before the pausing occurs. It also seems to cause longer loading times on my system. However, a lot more memory is freed up in scene changes.
Elita, is there any point to enabling a Direct 3D flag in when this game is specifically OpenGL? In the acutal launcher line you can see there's a specific -Dsun.java2d.d3d=false command that you are overriding.
Sat, 11/12/2011 - 12:12
#124
I thought Opengl is only used
I thought Opengl is only used on Mac's version.
Again, you are free to comment on the profiles. The idea is that we create a good sets of profile and make public to everyone, and save people's time.
I feel these profiles should be provided by OOO. But at least they are listening and slowly untying our hands. Who knows maybe they are ahead of us and will provide a list of recommended profiles for people to use.
Sat, 11/12/2011 - 14:21
#125
Significant timing
Time plays a major role in my choice of extra.txt parameters.
I usually don't play for more than 2-3 hours in one session which, without tweaking, was enough to make simply chatting in Haven difficult. I'm aware of the longer loading times, but I believe it's a fair tradeoff for pauseless gameplay during the time that I do spend playing.
Speaking of timing, currently, the version of getdown available to players does not handle overriding default parameters. JVM itself might do something if it encounters the same parameter key multiple times with different values , but I couldn't find specifics. I assume it takes the first instance and ignores the rest. If history is a good predictor of events to come, expect overrides to magically start working on Monday-ish.
Finally, Eltia, you should do some timing! Not now, of course, the time is not right. Maybe around Monday afternoon. Do some profiling of -Dsun.java2d.d3d=true compared to using the default. By observation, we can see what software the game uses to interface with the graphics hardware, but we can't tell anything about the specific implementations. I have my doubts that the java2d library is better than lwjgl (or OOO's own code) otherwise everyone would already be using it. I have more doubts about Direct3D providing better performance than OpenGL in this use case. I have no evidence to support these doubts, however, so I would love to be proven wrong.
Sat, 11/12/2011 - 22:53
#126
Some mixed results
Good news is that, I tested my extra.txt on Shadow Lair. JVM used about 850MB of RAM (private + shared data) and it only lagged at the beginning of 2nd level of Ultra Snabby.
I also tried to replace the d3d options with the option "-Dsun.java2d.opengl=true", I do not notice any significant difference in performance. Looking at GPU loads, it's hover at around 12-13%. (Resolution was 1080p w/ max graphic setting.) I suspect none of the graphic acceleration option works.
I think the easiest way is for OOO to tell us what graphic libraries they use in SK and we just tweak that library's options.
Sun, 11/13/2011 - 13:49
#127
Another update
One of my guild members who play SK on Mac, has tested out some setting. According to him, SK won't boot with the incremental garbage collector (the Xms and Xmx options work though). So all he did was set minimum heap to 512MB and maximum heap to 1GB. He said that's already a big improvement.
I continue to see little to no difference between D3D and OpenGL acceleration on Windows. I'm suspecting SK never use the library calls that are being accelerated (or SK turns them off by default). If someone get these acceleration working, please post.
Sun, 11/13/2011 - 16:23
#128
Some more Notes:
I'm usually playing the game with only the -Xmx1g variable added in my extra.txt and detail settings maximized at a windowed resolution of 1360x768 on a 1680x1050 screen. I'm on Win7 x64 with 64-bit JRE7u1, so server mode is enabled by default. I'm using a character idling just underneath the fire in the Rescue Camp instance that is nearly empty for all observations. Memory valuse may be affected by how many knights were around before I switched to a quiet instance. If you really want to know full PC details just ask, but they are being held constant for each test state anyway. If I don't state an -Xmx variable it's assumed 1g.
Default State:
FPS: 100
GPU load: 25%
CPU load: 26%
Working Set Memory: 394mb (default) / 564mb (-Xmx1g)
-Dsun.java2d.opengl=true
FPS: 94
GPU load: 25%
CPU load: 26%
Working Set Memory: 633mb
-Dsun.java2d.d3d=true
FPS: 100
GPU load: 25%
CPU load: 26%
Working Set Memory: 576mb
Corsa's Setup
FPS: 95
GPU load: 25%
CPU load: 26%
Working Set Memory: 362mb
Corsa's Setup + -Dsun.java2d.opengl=true
FPS: 90
GPU load: 24%
CPU load: 26%
Working Set Memory: 365mb
My initial reaction is that trying to change the render path either has no or negative results on my system. I'm still very skeptical of anything involving Direct 3D since this game seems to avoid anything proprietary to a single operating system brand. Corsa's summed tweaks also slowed down the game overall but really dropped memory usage. Though if I've got the resources to keep things in the memory I'd personally rather use it. Then again, remember that by default on my system -XX:+UseParallelGC is already enabled as a GC tuning option since it's considered server class. That could be giving a nice boost. If anyone wants me to another option looked into I can give it a whirl on my computer for consistency's sake.
Also, remember that on any system the default flags are (as taken from lancher.log):
-Xmx256M
-XX:+AggressiveOpts
-XX:SoftRefLRUPolicyMSPerMB=10
-Dsun.java2d.d3d=false
Sun, 11/13/2011 - 16:11
#129
@Eltia
I m on a mac and
@Eltia
I m on a mac and really try to improve the performance... what was the text file your guild member used on his mac ? Could u post ? is he on lion or leopard ?
Thanks
Sun, 11/13/2011 - 16:22
#130
Elita linked to a list of
Elita linked to a list of extra.txt files, one of them was an experimental Mac one. It contains the following:
-Xms512M
-Xmx1G
-Dsun.java2d.opengl=true
Remember that if you're on a system with around 2GB of physical memory, the -Xmx1G may be a bit too high.
Sun, 11/13/2011 - 19:13
#131
@Dogrock
I noticed very early on in this thread, you claimed that SK only uses OpenGL. I am wondering if you can provide a source to that claim?
It sounds right that OOO didn't do anything useful with platform specific / proprietary extension. I wonder what API they used to program the 3D stuff in SK.
Also, is there a way built-in the game to show FPS? Or we have to pipe the game through FRAP?
Sun, 11/13/2011 - 20:36
#132
I can't provide you with a
I can't provide you with a concret "he said this graphics renderer" as a proof. From the time I've spent in the Technical Issues forum, I constantly see messages relating to OpenGL. There's some more clear evidence involving the use of the LWJGL, but that information is not to be discussed here. I also don't see a reason for specifying d3d false otherwise or focusing on a technology only avalible to one of the three supported platforms.
I use FRAPS as my defacto program for collecting framerate data.
Mon, 11/14/2011 - 06:19
#133
Can someone help me?
I'm on
Can someone help me?
I'm on an iMac (with the graphics card ATI Radeon 2600 something) and I've been trying almost everything I've read in this thread.
First, I tried using the Mac version of extra.txt. I noticed that when on Haven, the graphics is stuttering and some images are missing. I removed it.
Second, I tried using the Windows methods. I'm using Windows 7 32-bit on Bootcamp. The Very High and Maximum extra.txt (even the Corsa version) gives me the stutters. Not only that. I noticed red dots starting to appear on my desktop (those red dots like you're overclocking a graphics card and when it can't handle things anymore they start to appear.)
I'm really lost here. Hope to get some sort of help to get rid of lag (when on Clockworks I often see other knights going beyond the walls)
Here's my setup:
4GB memory
Bootcamp side:
Windows 7 32-bit
Steam user
Mac side:
Steam user
iMac 2007 version
Mon, 11/14/2011 - 06:56
#134
@Porky Pig
How s your playing
@Porky Pig
How s your playing experience on a Imac ? I ve send numerous mail to OOO because I have lots of stuttering on my imacs that do not have OSX Lion on it....my specs are way above minimum requirements too...they couldnt help me at all...
I have 2 imacs, the one running on Lion works fine, what s your OSX ?
Mon, 11/14/2011 - 07:15
#135
@Tontontat
When playing on the Mac, I still experience lag. Best example is the Iron Munitions Factory and doing Shadow Lairs. Heck, even doing the standard FSC lags me up.
Lag types I usually encounter:
- knights / zombies going through the walls
- my chat message come up after 2-3 seconds
- running for the Blackstone Bridge's first set of wheels gets me on fire (when I'm positive I've cleared them)
- running for a bit then full-stopping will make my knight jump to a different location, around 2-3 feet
I'm running Mac OSX 10.7.2 Lion.
Mon, 11/14/2011 - 10:05
#136
@Dogrock. It's good to know
@Dogrock. It's good to know that my assumption is correct. (That your claim SK only uses OpenGL is a speculation.) It would be very kind if OOO can step up and provides a clear statement on what graphical API exactly does SK use. This way we won't have to guess and waste effort on the wrong path.
@Porky-Pig. The symptoms you described do not look good. The "Red Dots" you described are graphic artifacts that, I am guessing it's hardware or driver related. My advice would be to do a more thorough check on your hardware and driver first, before attempting any of the tweaks in this thread.
Mon, 11/14/2011 - 16:49
#137
@Endymion
The extra.txt file doesn't seem to work properly. The default Xmx256M option that the game launches with doesn't get overriden with the options I put in the file. For example, I used Xmx600M and Xms400M and when I inspect the process with Java VisualVM all the three options seem to be active for the jvm: Xmx600M, Xms400M and Xmx256M. Since Xmx256M was passed first it's the one actually working, so the jvm ignores the other two. I'm on linux and I've been able to properly set the heap sizes using a custom launcher.
Mon, 11/14/2011 - 16:54
#138
It would be very kind if OOO
It would be very kind if OOO can step up and provides a clear statement on what graphical API exactly does SK use. This way we won't have to guess and waste effort on the wrong path.
Spiral Knights uses OpenGL, via LWJGL (http://lwjgl.org/), on all platforms. The various sun.java2d options are not likely to have any effect on performance. The reason the startup parameters in getdown.txt disable D3D acceleration (-Dsun.java2d.d3d=false) is because we found that it can interfere with OpenGL initialization (causing crashes on Windows). The easiest way to tweak graphics settings for OpenGL is through your video driver settings (which typically allow you to override texture filtering modes, antialiasing, etc.)
As a note, the next release will attempt a max heap size of 512m (-Xmx512M) and fall back to the earlier value (-Xmx256M) if the JVM can't initialize due to insufficient memory.
Mon, 11/14/2011 - 17:14
#139
Since Xmx256M was passed
Since Xmx256M was passed first it's the one actually working, so the jvm ignores the other two. I'm on linux and I've been able to properly set the heap sizes using a custom launcher.
What version of Java are you using? In every case I've tried, the option that comes last on the command line is the one used.
Here's a program that simply prints out the maximum memory:
public class Test {
public static void main (String... args) {
System.out.println(Runtime.getRuntime().maxMemory()/1024/1024);
}
}
And the corresponding output (Ubuntu Linux, Sun/Oracle Java v1.6.0_26):
andrzej@arafura:/export/projectx$ java -mx64M -mx128M Test
113
andrzej@arafura:/export/projectx$ java -mx128M -mx64M Test
56
andrzej@arafura:/export/projectx$ java -Xmx256M -Xmx600M -Xms400M Test
533
Tue, 11/15/2011 - 01:19
#140
Thank you for the explanation of extra.txt file. The game is now able to utilize more memory than the 512MB limit and I've experienced a smoother gameplay with very minimal pauses (almost non-existant).
Tue, 11/15/2011 - 07:39
#141
@Andrzej
Thank you for your clarification. The necessary updates have been made to the sample profiles.
Tue, 11/15/2011 - 14:26
#142
@Andrzej
My bad, it's working fine :) Thanks for the tip.
I also want to add that Corsa's configuration has greatly improved my gameplay.
Tue, 11/15/2011 - 15:11
#143
ermmm couldnt find the : Mac
ermmm couldnt find the : Mac Web Client - ~/Library/Application Support/spiral/
Wed, 11/16/2011 - 17:25
#144
Different Garbage Collections
I've tested alot around and also tried different Garbage Collectors (GC) as well as different other settings.
I've had big problems with the standard configuration, like every 30 minutes a 1 second freeze in arena. I died like 1000 times from this. And now after some investigation with VisualVM and some configurations I come up to some solutions.
Read this to an end, because there are some informations you should know.
I'm using the Sun JVM, and I think the most people, too, since it is the first result when searching for java. When it is stated as "better for Multicores" it better to have at least 3 Cores, with 2 Cores it doesn't really matters).
Some short rules:
- Don't use "-Xincgc" with this gc solutions or mix the gc solutions.
- Steam doesn't use you installed JVM, instead it has it's own, a really old one.
- If using G1 make sure to have Sun JVM 1.7 or above in the java_vm directory in you steam spiral knights directory.
- Don't overdo it with RAM and the disableGC option, else you could have a big surprisingly freeze.
GC Solution 1 - CMS:
This is the Concurrent Mark Sweep GC. It's not running too long, so you just have small freezes, like 100-200 milliseconds. It checks incrementally also the Tenured Generation and tries to reduce it, so it works against a Full Collection. Still when a Full Collection is done it freezes. This is recommended for single core CPUs, since Parallel can't help then.
-XX:+UseConcMarkSweepGC : Setting it as GC
-XX:+UseParNewGC : Adding a parallel GC for Young Generation Objects (better for Multicores) (specially for CMS, don't use it with other GCs)
-XX:+CMSIncrementalMode : Decreases the freeze time again, by doing the concurrent phase incremental.
-XX:+CMSIncrementalPacing : Enables automatic adjustment of the incremental mode above, based on statistics collected at runtime.
GC Solution 2 - Parallel:
This should be the standard configuration when using Sun JVM 1.5 or above, except for the UseParallelOldGC option. For each core it creates one thread for the GC, more than 1 thread per core are decreasing the performance, so you shouldn't do it. So you don't have to adjust the number of threads for this GC.
It still just freezes when doing collections, but it should do them much more faster. I tested it in FSC when I got constantly freezes with default configuration and it worked for me like a wonder, it reduced the occurence rate by like more than 90%. If you have a good multicore CPU try this.
-XX:+UseParallelGC : Setting a parallel GC for Young Generation. (better for Multicores)
-XX:+UseParallelOldGC : Setting also a parallel GC for the Tenured Generation. (better for Multicores)
GC Solution 3 - G1:
This one is still in experimental phase (like a Garbage Collector in beta phase) so use it with care. It only works when having a SUN JVM 1.7 or above, else SK just won't start with this options. It used a lot more of your CPU but does also a good job in reducing freeze time for me. I use this as my standard configuration. This GC can't be explained with Young and Tenured Generation Collections since it is a completly new method of garbage collection.
I can say it frees constantly the RAM and Full Collections occur as often as being hit from a thunderbolt.
-XX:+UnlockExperimentalVMOptions : Enabling experimental mode in JVM
-XX:+UseG1GC : Setting G1 for GC.
-XX:MaxGCPauseMillis=50 : Setting the maximum freeze time in milliseconds (still intervall has to be set)
-XX:GCPauseIntervalMillis=1000 : Setting the intervall in which the above maximum millicesonds are allowed to occur.
So in this example the G1 tries to run for 50 milliseconds in one second. Setting this to 0 doesn't mean you won't get freezes ... btw 0 is a bad number for options.
Extra Solutions:
Explicit Collections
To adjust the Explicit Garbage Collections which comes from the application (so from OOO devs) you have 2 options.
-XX:+DisableExplicitGC : This disables them completly, so nothing is done then the application commands a GC.
Only when using CMS:
-XX:+ExplicitGCInvokesConcurrent : This invokes a Concurrent GC for the CMS instead of a Full Collection.
Since I use G1 I just can turn em off, so I use the disable option. Since I don't like freezes.
Decreasing Loading Time
I have found an option which really does a good job on that.
-Xnoclassgc : This never collects classes, so they remain in Permanent Generation, which is about 20MB and doesn't really grow much bigger. No negative effects until now.
Garbage Collection explained:
All the RAM for the application is divided into the Young Generation and the Tenured Generation, and also the Permanent Generation. The Garbage Collection frees RAM from objects which is not used anymore from the application. When a GC occurs the application is completly stopped, in default configuration.
Young Generation: All new created objects are firstly stored there (in default configuration (from OOO) it's about 60 MB) and it fill very fast like, 15 MB/sec, just by standing around in town. When it is full, a GC starts to clear the Young Generation and stores the remaining objects which are still alive in the game (referenced) in one 1 of 2 special parts of the Young Generation (Survivor Space). If the Survivor Space is full it is also garbage collected and still living objects are copied into the Tenured Generation. This GCs are mostly not even mentioned if the Collections are not too big, like 100 MB is ok.
Tenured Generation: The longer living objects are stored here. It's about 170 MB in the default configuration of SK. If it is full a GC occurs and it can take much longer than in the Young Generation and causes the big freezes. I heard of 10 second freezes from some people. On my hardware it causes around 1-2 second freezes. When many new Objects are created, like entering a new level or going through the arena this Generation increases fast. If the Young Generation GC Rate increases (e.g. cause of much monsters and battle in Game) this Generation increases too fast with young living objects, which doesn't really belong here, but they didn't had enough time to die while beeing in Young Generation. Increasing this part will deacrease the rate at which the freezes occur, but the freezes will get worse, since the GC has more to RAM to clear.
So Until now we know of 3 Situations when GC is invoked.
When Young Generation is full, a GC is started for the Young Generation and stores the survivors in Survivor Space.
When Survivor Space is full, a GC is started for Survivor Space and stores the survivors of this GC in Tenured Generation. This occurs when Young Generation is full. So 2 GCs are done in a row.
When Tenured Generation is full, a GC is started for the Tenured Generation. This occurs when objects are stored from Survivor Space, so 3 GCs are done in a row. This is known as Full Collection.
But there is still another Situation when a GC occurs. That is when the applications commands it. Then an Explicit Collection is done, this is like a Full Collection but on command, so it does not check wether it is needed or not. This is done very often in SK.
Now I hope you have learned something and you know better what you are doing there with your options.
Just increasing RAM with some GC could make the freezes worse, so better test it and test it for more than just running through Haven.
Now all* my wisdom belongs to you.
*not really
Wed, 11/16/2011 - 18:43
#145
Question for high end rigs
Ok i've gone thru this thread and i was just wondering, i have a i7 920@3.1ghz/6gig corsair dominator xmp settings/560ti vid card, will someone with a higher end rig notice a difference by doing the extra.txt config? Everything atm is totally smooth for me with no lag/stuttering.
Wed, 11/16/2011 - 19:14
#146
@Knightmehr
Thank you for sharing these information. I have put up two more profiles for the parallel GC and G1 GC (with credit attribute to you). Feel free to check them out.
Wed, 11/16/2011 - 22:37
#147
Overkill
This is probably my own fault for spending so much time blogging about GC options without saying enough about why I was looking into it - especially within the last week. My advice is to forget that all options except -Xmx exist. I recommended GC tuning at a time when I thought overriding the maximum heap size was impossible.
As Andrzej stated in #138, new defaults were issued with the Nov. 16, 2011 update. I think these new settings should be sufficient in combating most if not all of the performance issues we've been discussing in this thread. Dogrock's experiments (#104, #128, etc.) support this claim.
Thu, 11/17/2011 - 08:52
#148
Some results
-Xnoclassgc works wonders. Map loading is now almost instant.
The Java 7's G1 also seems to work very well. The drawback seems to be that it uses more CPU cycles.
One of my guildmates who isn't into tweaking, just played the game with the new OOO tweaked setting last night. He reported the game appears to be smoother now.
I noticed my nVidia driver has stopped applying Antialiasing to SK and I couldn't get it to work anymore (even with nVidia Inspector). Is it just me or other people observing the same thing?
Thu, 11/17/2011 - 08:57
#149
hey, the extra file keeps
hey, the extra file keeps getting deleted. I'm using steam. Sometimes, when i check the SK folder, it's just gone, I dont know what triggers this.
Any ideas?
Thu, 11/17/2011 - 09:57
#150
@Juances
Copy the file to Steam SK folder before you start Steam.
im with kirnan, why not just post a compiled binary. if spiral-spy didnt get ban-hammered i doubt the stub launcher will.