Openfire Memory Leak

[tmark]

http://java-monitor.com/postedimages/e2238bd6-7809-4466-a5e4-0521c20ed688.png

Hello,

We have a Windows VM running openfire 3.6.4 and are having major problems with our java memory. The server keeps running out of java memory, and then takes over all memory on the server and freezes and has to be rebooted. We have about 450 concurrent users.

I attached a graph of the heap memory. You can see at the end of the graph I had to restart as users weren't receiving messages or able to log into conference rooms. I've looked on the openfire forums and here and haven't found anything I've been able to do that has helped. I'm not real familiar with these kinds of java issues so any help is appreciated.

Thanks,
tyler

[kjkoster]

Dear Tyler,

Check out this thread on Ignite Realtime where another MUC user has a similar issue: http://community.igniterealtime.org/message/208938#208938

A user name 'mikeycmccarthy' posts the following that might be of interest to you:

Visit the 'Other Settings' section for your MUC conference. We found that the combination of flush interval and batch size were causing us problems with memory leaks. From what we could work out from the code, everytime someone writes a message in the room it is held in memory until that interval comes around. When it writes it does it in a batch of the specified size. If there are loads of messages going back and forth (as there were in our load test) the backlog of messages to write kept on increasing and increasing till we ran out of memory.

I don't know if this is a problem that only surfaces with load tests as I'd assume in quiet periods it would eventually calm down and clear down the log. We've changed to flush interval 20 and batch size 500 and haven't seen problems since.

Not sure this will solve your problem. The original poster never followed up with the effect of the changes for him, but it may be worth a shot. Let s know how you fare.

Kees Jan

[tmark]

Thanks for the follow up! I did try changing the conference room settings, no such luck. I've also put in the system property cache.username2roster.maxLifetime = 419430400 and xmpp.pep.enabled = false to see if that would help. None of these changes seem to have helped. I'm about ready to put the 3.7 beta version in production as I don't see any other way around it.

[Panzertax]

Hi

I have the same problem. When my room has more then 500 users it gets the same graph as you.
Did you have any luck installing 3.7 beta?

[tmark]

For some reason mine is working again. It has been up for over 11 days and actually looks like it is collecting garbage and managing correctly. I changed so many settings that I am not sure which one worked, so I will give you a rundown of what I did:

1) Added system property -- cache.username2roster.maxLifetime = 419430400

2) Added system property -- xmpp.pep.enabled = false

3) Under Group Chat settings -- I changed the Idle User Settings to Kick users after thay have been idle for 30 minutes, and changed the conversation logging to a Flush interval (seconds): 300, and Batch size to 50.

I am not sure, but I think kicking the idle users was the big change that actually made a difference. Before I had it set to Never kick idle users. A disclaimer though, I have had complaints from users about getting kicked from conference rooms. But I figure getting kicked from a conference room you aren't using for 30 minutes beats having all my users getting kicked out 5 times a day! Let me know if any of these things work for you.

[Panzertax]

But did you install the new version?

1) Added system property -- cache.username2roster.maxLifetime = 419430400
2) Added system property -- xmpp.pep.enabled = false

I have added these.

3) Under Group Chat settings -- I changed the Idle User Settings to Kick users after thay have been idle for 30 minutes, and changed the conversation logging to a Flush interval (seconds): 300, and Batch size to 50.

I have disabled room logging so I guess this wont help me (but I changed them anyway)

I am not sure, but I think kicking the idle users was the big change that actually made a difference. Before I had it set to Never kick idle users. A disclaimer though, I have had complaints from users about getting kicked from conference rooms. But I figure getting kicked from a conference room you aren't using for 30 minutes beats having all my users getting kicked out 5 times a day! Let me know if any of these things work for you.
My problem is that my chat is used as a game room, where everybody will use the room in a < 2 hour intervall.

[tmark]

I did not install the new version. I do have it running on my test server right now and it seems ok. I still have 3.6.4 installed in production.

Did you change the Idle User Settings to kick after 30 minutes (or whatever you want it set at)?

[Panzertax]

Did you change the Idle User Settings to kick after 30 minutes (or whatever you want it set at)?

Yes I did this also. But I have 1000 concurrent users within a 2*45 soccer period.. But I set it to kick after 40 min. Ill see how it goes :)

Thanks

[kjkoster]

Guys,

Openfire 3.7.0 was officially released. (http://community.igniterealtime.org/blogs/ignite/2011/03/02/openfire-370-has-been-released) So you can try it without worrying about beta status. :)

Kees Jan

[Panzertax]

Hi

I think we have found a solution to this problem. Openfire comes with 32bit jre. We installed 64 bit jre and it seems to work better now.
So I would upgrade to 3.7.0 and install 64 bit java and see if this helps.