Heap usage 80% - take action?

[Kees de Kooter]

http://java-monitor.com/postedimages/82e1aa0b-cef7-40cf-8c01-895ba8ff0a18.png

Should I increase the heap size for this tomcat instance or should I take other measures?

[kjkoster]

Dear Kees,

Does this server handle traffic all trough the day, or it it processing batches?

Do you experience any pauses in processing? If you do, they might mean GC blocking other processing. Could you also post the GC collection time graph? That will show you what GC overhead you have.

What you could try as an experiment is to increaes the size of your new genertion, to see if that helps. Maybe you have a lot of short-lived objects that end up in your old generation memory pools. Making the new generation larger might help.

Kees Jan

[kjkoster]

Dear Kees,

Oh, and if everything runs smoothly, there probably is nothing to worry about. :)

Kees Jan

[Kees de Kooter]

Hi Kees Jan,

It is a search engine on a public website. A big batch runs at 2 AM (lucene indexing), a small one at 6 AM (sitemap building).

Client is complaining of poor performance from time to time during the day.

GC graph is posted in a new thread.


I suppose I need to set the -XX:NewRatio switch on the JVM. What is the default value for that and what do you sugggest?

[kjkoster]

Dear Kees,

Hmm. We're off into shadow-land here. Sun's documentation contradicts itself on what the default values are. The tuning also depends on what GC is in use.

Looking at the GC time graph, I see some peaks that go up to several seconds, even during the day. Do they coincide with the performance problems that your client is experiencing?

Have a look at what Peter Johnson has to say about the new pool (http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4172629#4172349).

Please bear in mind that the problem might be something else entirely. Have you been able to observe the problems that your client desribes?

What you might want to consider doing is enable your server's HTTP response time logging (http://java-monitor.com/forum/showthread.php?t=5) and start using something like jamon or p6spy to track database performance.

Kees Jan

[kjkoster]

Dear Kees,

Did you make any headway on this problem? I'm curious if you managed to pin this to a particular subsystem yet and if so, how?

Kees Jan

[Kees de Kooter]

Hi Kees Jan,

I have enabled response time logging and added the following jvm flags, taken from Peter Johnson's presentation:

-XX:NewSize=256m -XXMaxNewSize=256m

Now I am going to watch the java monitor graphs ;-)

[kjkoster]

Dear Kees,

Which reminds me, if this is backed by a MySQL database it may pay to enable slow query logging as well. The default threshold is 5 seconds, which I usually lower to 1 second.

Kees Jan

[kjkoster]

Dear Kees,

I have two graphs from a relatively quiet server. They show that going up to 80% heap is fine, as long as your gc does not take too long.

Mine shows a gc time of just under half a second, which I find acceptable.

http://java-monitor.com/postedimages/91640d1b-59c6-46f6-9e71-e488add601e0.png


http://java-monitor.com/postedimages/0a3f9812-c292-4c76-80af-5732b2dff9f8.png


Hope this helps.

Kees Jan

[Kees de Kooter]

I can accept GC times of half a second. How can I determine the actual GC time on my server?

[kjkoster]

Dear Kees,

You can see a graph of that time on the host's page, in the section marked 'Garbage Collection'. Look at the second graph, named 'CollectionTime'.

Kees Jan

[kjkoster]

The time is in milliseconds.