Wednesday, June 10, 2009

Getting Java stack trace from Glassfish

If you run into problems with Glassfish (e.g. Glassfish hangs) and need to get a stack trace of the java process, here's how you do it without using jstack:

1. Start glassfish using asadmin

mclaren> asadmin stop-domain
Domain domain1 stopped.
mclaren> asadmin
Use "exit" to exit and "help" for online help.
asadmin> start-domain
Starting Domain domain1, please wait.
Default Log location is /home/affinium7/mas/tmp/glassfish/domains/domain1/logs/server.log.
Redirecting output to /home/affinium7/mas/tmp/glassfish/domains/domain1/logs/server.log
Domain domain1 is ready to receive client requests. Additional services are being started in background.
Domain [domain1] is running [Sun GlassFish Enterprise Server v2.1 (9.1.1) (build b60e-fcs)] with its configuration and logs at: [/home/affinium7/mas/tmp/glassfish/domains].
Admin Console is available at [http://localhost:4848].
Use the same port [4848] for "asadmin" commands.
User web applications are available at these URLs:
[http://localhost:8888 https://localhost:8181 ].
Following web-contexts are available:
[/web1 /__wstx-services ].
Standard JMX Clients (like JConsole) can connect to JMXServiceURL:
[service:jmx:rmi:///jndi/rmi://mclaren:8686/jmxrmi] for domain management purposes.
Domain listens on at least following ports for connections:
[8888 8181 4848 3700 3820 3920 8686 ].
Domain does not support application server clusters and other standalone instances.



2. Remain in asadmin (do not exit)

3. Open another terminal window and get the process id of the running domain
(normally this is the process running com.sun.enterprise.cli.framework.CLIMain)


mclaren> ps -ef| grep glass
affiniu7 479618 565548 120 09:44:38 pts/8 27:47 /usr/java5_64/jre/../bin/java -Dcom.sun.aas.instanceRoot=/home/affinium7/mas/tmp/glassfish/domains/domain1 -Dcom.sun.aas.ClassPathPrefix= -Dcom.sun.aas.ClassPathSuffix= -Dcom.sun.aas.ServerClassPath= -Dcom.sun.aas.classloader.appserverChainJars.ee= -Dcom.sun.aas.classloader.appserverChainJars=admin-cli.jar,admin-cli-ee.jar,j2ee-svc.jar -Dcom.sun.aas.classloader.excludesList=admin-cli.jar,appserv-upgrade.jar,sun-appserv-ant.jar -Dcom.sun.aas.classloader.optionalOverrideableChain.ee= -Dcom.sun.aas.classloader.optionalOverrideableChain=webservices-rt.jar,webservices-tools.jar -Dcom.sun.aas.classloader.serverClassPath.ee=/lib/hadbjdbc4.jar,/home/affinium7/mas/tmp/glassfish/lib/SUNWjdmk/5.1/lib/jdmkrt.jar,/lib/dbstate.jar,/lib/hadbm.jar,/lib/hadbmgt.jar,/lib/mfwk_instrum_tk.jar -Dcom.sun.aas.classloader.serverClassPath=/home/affinium7/mas/tmp/glassfish/lib/install/applications/jmsra/imqjmsra.jar,/home/affinium7/mas/tmp/glassfish/imq/lib/jaxm-api.jar,/home/affinium7/mas/tmp/glassfish/imq/lib/fscontext.jar,/home/affinium7/mas/tmp/glassfish/imq/lib/imqbroker.jar,/home/affinium7/mas/tmp/glassfish/imq/lib/imqjmx.jar,/home/affinium7/mas/tmp/glassfish/lib/ant/lib/ant.jar,/home/affinium7/mas/tmp/glassfish/lib/SUNWjdmk/5.1/lib/jdmkrt.jar -Dcom.sun.aas.classloader.sharedChainJars.ee=appserv-se.jar,appserv-ee.jar,jesmf-plugin.jar,/lib/dbstate.jar,/lib/hadbjdbc4.jar,jgroups-all.jar,/lib/mfwk_instrum_tk.jar -Dcom.sun.aas.classloader.sharedChainJars=javaee.jar,/usr/java5_64/jre/../lib/tools.jar,install/applications/jmsra/imqjmsra.jar,com-sun-commons-launcher.jar,com-sun-commons-logging.jar,/home/affinium7/mas/tmp/glassfish/imq/lib/jaxm-api.jar,/home/affinium7/mas/tmp/glassfish/imq/lib/fscontext.jar,/home/affinium7/mas/tmp/glassfish/imq/lib/imqbroker.jar,/home/affinium7/mas/tmp/glassfish/imq/lib/imqjmx.jar,/home/affinium7/mas/tmp/glassfish/imq/lib/imqxm.jar,webservices-rt.jar,webservices-tools.jar,mail.jar,appserv-jstl.jar,jmxremote_optional.jar,/home/affinium7/mas/tmp/glassfish/lib/SUNWjdmk/5.1/lib/jdmkrt.jar,activation.jar,appserv-rt.jar,appserv-admin.jar,appserv-cmp.jar,/home/affinium7/mas/tmp/glassfish/updatecenter/lib/updatecenter.jar,/home/affinium7/mas/tmp/glassfish/jbi/lib/jbi.jar,/home/affinium7/mas/tmp/glassfish/imq/lib/imqjmx.jar,/home/affinium7/mas/tmp/glassfish/lib/ant/lib/ant.jar,dbschema.jar -Dcom.sun.aas.configName=server-config -Dcom.sun.aas.configRoot=/home/affinium7/mas/tmp/glassfish/config -Dcom.sun.aas.defaultLogFile=/home/affinium7/mas/tmp/glassfish/domains/domain1/logs/server.log -Dcom.sun.aas.domainName=domain1 -Dcom.sun.aas.installRoot=/home/affinium7/mas/tmp/glassfish -Dcom.sun.aas.instanceName=server -Dcom.sun.aas.processLauncher=SE -Dcom.sun.aas.promptForIdentity=true -Dcom.sun.enterprise.config.config_environment_factory_class=com.sun.enterprise.config.serverbeans.AppserverConfigEnvironmentFactory -Dcom.sun.enterprise.overrideablejavaxpackages=javax.help,javax.portlet -Dcom.sun.enterprise.taglibs=appserv-jstl.jar,jsf-impl.jar -Dcom.sun.enterprise.taglisteners=jsf-impl.jar -Dcom.sun.updatecenter.home=/home/affinium7/mas/tmp/glassfish/updatecenter -Ddomain.name=domain1 -Djava.endorsed.dirs=/home/affinium7/mas/tmp/glassfish/lib/endorsed -Djava.ext.dirs=/usr/java5_64/jre/../lib/ext:/usr/java5_64/jre/../jre/lib/ext:/home/affinium7/mas/tmp/glassfish/domains/domain1/lib/ext:/home/affinium7/mas/tmp/glassfish/javadb/lib -Djava.library.path=/home/affinium7/mas/tmp/glassfish/lib:/home/affinium7/mas/tmp/glassfish/lib:/home/affinium7/mas/tmp/glassfish/lib -Djava.security.auth.login.config=/home/affinium7/mas/tmp/glassfish/domains/domain1/config/login.conf -Djava.security.policy=/home/affinium7/mas/tmp/glassfish/domains/domain1/config/server.policy -Djava.util.logging.manager=com.sun.enterprise.server.logging.ServerLogManager -Djavax.management.builder.initial=com.sun.enterprise.admin.server.core.jmx.AppServerMBeanServerBuilder -Djavax.net.ssl.keyStore=/home/affinium7/mas/tmp/glassfish/domains/domain1/config/keystore.jks -Djavax.net.ssl.trustStore=/home/affinium7/mas/tmp/glassfish/domains/domain1/config/cacerts.jks -Djdbc.drivers=org.apache.derby.jdbc.ClientDriver -Djmx.invoke.getters=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Xmx512m -cp /home/affinium7/mas/tmp/glassfish/lib/jhall.jar:/home/affinium7/mas/tmp/glassfish/lib/appserv-launch.jar:/home/affinium7/mas/tmp/glassfish/domains/domain1/lib/log4j com.sun.enterprise.server.PELaunch start
affiniu7 508166 610564 0 10:11:12 pts/6 0:00 grep glass
affiniu7 565548 729806 0 09:44:27 pts/8 0:15 /usr/java5_64/jre/../bin/java -Dcom.sun.aas.instanceName=server -Djava.library.path=/home/affinium7/mas/tmp/glassfish/lib:/home/affinium7/mas/tmp/glassfish/lib:/home/affinium7/mas/tmp/glassfish/lib -Dcom.sun.aas.configRoot=/home/affinium7/mas/tmp/glassfish/config -Djava.endorsed.dirs=/home/affinium7/mas/tmp/glassfish/lib/endorsed -Dcom.sun.aas.processLauncher=SE -cp /home/affinium7/mas/tmp/glassfish/javadb/lib/derby.jar:/home/affinium7/mas/tmp/glassfish/jbi/lib/jbi-admin-cli.jar:/home/affinium7/mas/tmp/glassfish/jbi/lib/jbi-admin-common.jar:/home/affinium7/mas/tmp/glassfish/lib:/home/affinium7/mas/tmp/glassfish/lib/comms-appserv-rt.jar:/home/affinium7/mas/tmp/glassfish/lib/appserv-rt.jar:/home/affinium7/mas/tmp/glassfish/lib/appserv-ext.jar:/home/affinium7/mas/tmp/glassfish/lib/javaee.jar:/home/affinium7/mas/tmp/glassfish/lib/appserv-se.jar:/home/affinium7/mas/tmp/glassfish/lib/comms-appserv-admin-cli.jar:/home/affinium7/mas/tmp/glassfish/lib/admin-cli.jar:/home/affinium7/mas/tmp/glassfish/lib/appserv-admin.jar:/home/affinium7/mas/tmp/glassfish/lib/commons-launcher.jar:/home/affinium7/mas/tmp/glassfish/lib/install/applications/jmsra/imqjmsra.jar -Dcom.sun.appserv.admin.pluggable.features=com.sun.enterprise.ee.admin.pluggable.EEClientPluggableFeatureImpl com.sun.enterprise.cli.framework.CLIMain


4. Do a kill -3

mclaren> kill -3 565548

5. Switch back to asadmin terminal and you will see the location of the stack trace:

asadmin> JVMDUMP006I Processing Dump Event "user", detail "" - Please Wait.
JVMDUMP007I JVM Requesting Java Dump using '/home/affinium7/mas/tmp/glassfish/bin/javacore.20090610.101131.565548.txt'
JVMDUMP010I Java Dump written to /home/affinium7/mas/tmp/glassfish/bin/javacore.20090610.101131.565548.txt
JVMDUMP013I Processed Dump Event "user", detail "".


6. View the file to debug further :-)

No comments:

Post a Comment