java.lang.OutOfMemoryError: PermGen space

When you get an OutOfMemoryError with the message "PermGen space" (not to be confused with message "Java heap space"), this means the memory used for class definition is exhausted. Fortunately, most of the time, this is easy to fix.


Solution 1: Increase the size of "PermGen space"

If you have a Java process that uses a lot of classes (lots of jars) or if you have many applications deployed to your application container (Tomcat), you can allocate more memory to that "PermGen space" using the -XX:MaxPermSize VM argument. For instance, to allocate 256 MB of RAM to "PermGen space", use:

java -XX:MaxPermSize=256M ...

 

Solution 2: Restart your application container

You can get this error if you redeploy an application (webapp) several time without restarting your application container (like Tomcat). Most application containers support hot-redeployment, but class-loading is complex and sometimes old class definitions remain in memory. In that case, your best option is to get used to always restart your application container (Tomcat) after you deploy an application to it. It is easy and it fixes the issue.

 

Solution 3: Fix your class-loader leak

If none of the above works, you are in trouble. Seriously, unless you hacked the class-loading of the JVM or application container, you should not get that issue. Or maybe it is a bug in a library you are using or in your application container (Tomcat). You can try to upgrade to latest versions. If you hacked the class-loaders yourself, why did you do that? Unless you are developing a JVM or an application container, you should not have to do that.


 

New website for my software development company

Yay! I got a new website for my application development company. I just wanted to brag about it. It features more relevant information and a lean design powered by Bootstrap. It is now hosted on the reliable Amazon S3. If you need a software for your business, call us. We will discuss your project and we'll give you a free quote, see our pricing and application hosting packages. All the applications we develop run on desktop, tablet and smartphone out of the box.

Alright, enough bragging, back to work now!

VisualVM slow with heap dump files

One great feature of VisualVM is that it can read heap dump files. Heap dumps are useful to diangose memory leaks. See this post for more details about memory leaks and how to solve them. Another great feature of VisualVM is that you can read a huge heap dump file and VisualVm will consume a minimal amount of memory to do so. For instance, you will be able to read a 8 Gigabytes heap dump file with VisualVM running on a development workstation having only 2 Gigabytes of RAM. In order to achieve that, VisualVM will parse the heap dump file and will create a work file on disk in the default system temp folder (/tmp by default on Linux). In theory that's great, but in practice, VisualVM becomes painfully slow because it constantly have to do disk I/O's to process the information.

This behavior is even more frustrating if you happen to have a server with 12 Gigabytes of RAM available for you. A simple solution for that is to create a ramdisk and tell VisualVM to use that ramdisk as the tmp folder.

First, create the ramdisk. Here I am on a linux development server and I create a tmp folder in my home. Then I create (mount) the ramdisk in the tmp folder I just created:

jdemers@debian12:~$ mkdir /home/jdemers/tmp
jdemers@debian12:~$ sudo mount -t tmpfs none /home/jdemers/tmp

 Then I launch VisualVM and I modify the java.io.tmpdir VM arg that tells VisualVM where the system tmp folder is.

jdemers@debian12:~$ ./jdk1.6.0_20/bin/jvisualvm -J-Xmx1g -J-Djava.io.tmpdir=/home/jdemers/tmp

Now VisualVM is much much faster and I can investigate and find the root cause of that memory leak much faster.


Good Camel Games - Fun Multiplayer Games

We are pleased to announce the launch of another Web 2.0 application powered by OpCode Solutions: Good Camel Games - Fun Multiplayer Games. Good Camel Games provides real-time multiplayer games with public and private chat rooms. Games are free and are online; no need to pay or download anything. No registration is required, just enter the app and play with real people around the world, in real-time. On the client side, the app uses Google Web Toolkit (no flash, no plugin required). On the server side, we have Java back-end under Tomcat. What is fun about Good Camel Games it is that games are short and easy to play, so you can enjoy a game even if you only have 2 minutes.

USpeak - Anonymous Discussions

USpeak - Anonymous Discussions We are very pleased to announce the launch of USpeak. USpeak is a free service allowing anybody to create anonymous and confidential discussions. The discussions on USpeak are anonymous because all messages posted on a discussion are anonymous. The user who created the discussion itself also remains anonymous. However, all participants see who is participating in the discussion, they just don’t know who said what. Discussions are also confidential because a discussion is only visible to its participants. With USpeak, we hope to make it easier for people to talk about delicate, unpopular or taboo subjects. Here are few scenarios:

  • A corporation wanting its employees to give anonymous and interactive feedback about things that could be improved in the organization.
  • A family member expressing his/her views on a taboo subject.
  • An individual in a group wanting to change things and trying to create a momentum.

We firmly believe Freedom of Speech is for everybody and we hope USpeak will encourage people to freely speak their thoughts. For more information, see USpeak FAQ.

Solve java.lang.OutOfMemoryError: Java heap space

If you get an OutOfMemoryError with the message "Java heap space" (not to be confused with message "PermGen space"), it simply means the JVM ran out of memory. When it occurs, you basically have 2 choices:

1. Allow the JVM to use more memory using the -Xmx VM argument. For instance, to allow the JVM to use 1 GB (1024 MB) of memory:
java -Xmx1024m ...
2. Improve/Fix the application so that it uses less memory

In many cases, like in the case of a memory leak, the second option is the only sound choice. A memory leak happens when the application keeps more and more references to objects and never releases them. The garbage collector cannot collect those objects and the application will eventually run out of memory. At this point, the JVM will throw an OOM.

A memory leak can be very latent. For instance, the application might behave flawlessly during development and QA. However, it suddenly throws a OOM after several days in production at customer site. To solve that issue, you first need to find the root cause of it. The root cause can be very hard to find in development if it cannot be reproduced. Follow those steps to find the root cause of the OOM:

1. Start the application with the VM argument -XX:+HeapDumpOnOutOfMemoryError. This will tell the VM to produce a heap dump when a OOM occurs:
java -XX:+HeapDumpOnOutOfMemoryError ...
2. Reproduce the problem. Well, if you cannot reproduce in dev, you will have to use the production environment.
3. Use VisualVM to read the heap dump file and diagnose the issue.

The heap dump file has all information about the memory usage of the application. It allows you to navigate the heap and see what objects use memory and what references prevent the garbage collector from reclaiming the memory. This will give you very strong hints and you will (hopefully) be able to find the root cause of the problem. The problem could be a cache that grows indefinitely, a list that keeps collecting business-specific data in memory, a huge request that tries to load almost all data from database in memory, etc.

Once you know the root cause of the problem, you can elaborate solutions to fix it. In case of a cache that grows indefinitely, a good solution could be to set a reasonable limit to that cache. In case of a query that tries to load almost all data from database in memory, you may have to change the way you manipulate data; you could even have to change the behavior of some functionalities of the application.

If you do not want to wait for an OOM or if you just want to see what is in memory now, you can still generate heap dump. To manually trigger a heap dump, you have 2 choices:

1. Use VisualVM, right-click on the process on the left pane and select Heap Dump

2. If you do not have a graphical environment and can't use vnc (VisualVM needs a graphical environment), use jps and jmap to generate the heap dump file. Then copy the file to your workstation and use VisualVM to read the heap dump (File -> Load...):

user@host:~$ jps
20198
21734 WordFinder
21921 Jps
21168 Main

user@host:~$ jmap -dump:live,format=b,file=heap.bin 21734
Dumping heap to /home/user/heap.bin ...
Heap dump file created

You can also contact my application development company and I can personally help you with those kind of issues.

Here is a screenshot of VisualVM with a heap dump loaded:

Heap Dump in VisualVM 

Alternatively, you can also use jhat to read heap dump files.

Oracle to Acquire Sun

After IBM, now is the turn to Oracle. Oracle buys Sun and pays $7.4 billion for that acquisition ($9.50 per share), $5.6 billion if we take into account Sun's cash. Is that a good thing or a bad thing for the Java community? Probably a good thing as it will allow Sun's project to benefit from Oracle's technologies. But only future will tell.

With Java, GlassFish, MySQL, OpenOffice, Solaris and its cloud computing services, that certainly widens Oracle's possibilities. Hopefully Oracle will continue supporting adequately those famous open source projects. I am not concerned about Java, but maybe more about MySQL. While MySQL could certainly benefit from Oracle's expertise in database field, Oracle could also put MySQL behind in order to promote its more profitable flagship product: Oracle's RDBMS. The same is true for GlassFish and WebLogic. In any case, it is a very quick way for Oracle to have an open source offering for their products.

If we only consider Java, I have to admit that I rather see Oracle acquiring Sun than IBM doing it. The reason is simple, IBM already offers a Java runtime and they are competitors in that field. An acquisition by IBM would reduce the competition in that areas, which is always a bad thing for customers. In my opinion, Sun's JVM largely beats IBM's one, but it would still be a bad thing to see one of them die to the benefit of the other.

kill -3 is your friend

One nice feature of Java runtime is when you send the QUIT signal to a Java process, it outputs the full thread dump to stdout. To send the that signal, just open a terminal and type:

kill -QUIT <pid>

or

kill -3 <pid>

Where <pid> is the process Id. This does not terminate the process; all threads will continue doing what they were doing.

That feature can be very useful when the application seems to freeze or when you have a very intermittent issue (intermittent deadlock). With the full thread dump, you can see what every thread was doing at that particular moment. So in case of a deadlock, you will be able to see what monitors and what threads are involved.

This can also be helpful to diagnose performance bottlenecks. Suppose you are load testing an application and it does not deliver the expected throughput, but the CPU usage is not the problem. For instance, with kill -3 you will notice right away that the size of the jdbc connection pool is not big enough and all threads are waiting on it for a connection to free.

Remote Debugging in Java

One thing people ask me from time to time is how to debug a remote Java application. This can be very useful when you experience problems at customer site, but cannot reproduce them in a development environment. We all know logs files do not always contain all information required to solve the issues. In such case, remote debugging can be very useful.

To start remote debugging, one simply needs to add extra VM arguments to the Java command line:

-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8000

Then, you can use the remote feature of your IDE to connect to the Java process in using the port specified above (8000). For Instance, in eclipse, you would do:

Run -> Debug Configurations... -> Remote Java Applications -> Create new

In the Host field, you enter the host name or IP of the machine running the Java application. In the Port field, you enter the port specified above (8000).

Note that remote debugging also works nicely with SSH tunnels.

Opcode Blogs Now Launched

I am very pleased to announce the launch of the new Blogs site from Opcode Solutions. As a first release, the site contains one blog entirely focused on Java related topics: Expertises, Advices, Opinions, Tutorials, HOWTOs, News, all about Java Technologies. Opcode Blogs is powered by Apache Roller and runs under Tomcat with Sun's JVM on a Debian server.