Tomcat and java.net.SocketException: Too many open files

God bless monitoring. An hour ago I received a SMS from one of my servers that one of my sites was not available anymore. It is a dedicated server that runs 4 Tomcats in parallel. To track down the problem I started looking at the Tomcat logfiles. Whoo whats that? The fourth Tomcat was spamming logfiles like crazy. For the past four days it had created me 4 logfiles having a combined size of 600 GB. I was hitting a java.net.SocketException for too many open files.

First I thought I had a leak somewhere, which prevented files and sockets from getting closed properly. Actually this was not the main problem. Since all Tomcats were running as the same user, and I had not touched the open file limit for this user, the default maximum of 1024 in Ubuntu 9.10 server was way too little. I checked how many files I had open for this user.


ps aux | grep tomcat


Then for every PID I ran


lsof -p PID | wc -l


I had cleaned the logs and rebooted already. The combined result was that I already was scratching the 1000 mark for all Tomcats after rebooting. Very thin ice. To make a long story short, here is how to change the maximum open file limit on Ubuntu 9.10 server.

First you edit /etc/security/limits.conf and add your new limit for the user running Tomcat. In my case the user was called virtual:


virtual hard nofile 5120
virtual soft nofile 4096


In addition to that, edit the file /etc/pam.d/common-session and add


session required pam_limits.so


done! Reboot the machine, then verify the changes running


su virtual
ulimit -n

1 Kommentare:

Blogger hat gesagt…

DreamHost is one of the best hosting company for any hosting plans you require.