Thursday, March 11, 2010

Analyze JVM Logs

ystemOut.log and SystemErr.log files are text files so you can view them using simple text editor.
The Application server writes formatted messages to the SystemOut.log file and this is how a sample log message looks like

[7/2/09 9:12:22:645 PDT] 00000017 ApplicationMg A WSVR0221I: Application started: SchedulerCalendars

Each entry can be deciphered as follows

  • Time Stamp: The first part of the log message in sample code is [7/2/09 9:12:22:645 PDT]. It is the time stamp when the message was written. The time stamp is formatted using the locale of the process and it is 24 hour time stamp with milli-second precision

  • Thread ID: The next part in the log message is 00000017, which represents the thread id. The thread ID is an eight-character hexadecimal value that is generated from the hash code of the thread that issued the message

  • Short name: The short name is the abbreviated name of the component that issued the message. This name is typically the class name of a WAS component and would be some other identifier for the application. In our sample the ApplicationMg is component name

  • Event Type: The event type is a one character field that indicates the type of the message. The possible values are

    • F- Fatal message

    • E- Error message

    • W- Warning message

    • A- Audit message

    • I- Informational message

    • C- Configuration message

    • D- detail message

    • O- Messages that are written directly to System.out by an application or server component

    • R- Messages that are written directly to System.err by the user application or internal component.

    • Z- Place holder to indicate type was not recognized

    In my sample message A indicates this is audit message

Wednesday, March 10, 2010

.ear files deployment descriptor

The deployment descriptor of the EAR file is stored in the META-INF directory in the root of the EAR and is called application.xml. It contains information about the modules that makeup the application.

I have a HelloWorld.ear that contains one HelloWorldEJB module, which is a EJB project and HelloWorldWeb module which is a web module. The HelloWorldWeb module calls methods of HelloWorldEJB


<?xml version="1.0" encoding="UTF-8"?>
<application id="Application_ID" version="1.4" xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/application_1_4.xsd">
 <display-name>
 HelloWorld</display-name>
 <module id="EjbModule_1251662896937">
  <ejb>HelloWorldEJB.jar</ejb>
 </module>
 <module id="WebModule_1251663065906">
  <web>
   <web-uri>HelloWorldWeb.war</web-uri>
   <context-root>HelloWorldWeb</context-root>
  </web>
 </module>
</application>


In addition to the standard J2EE deployment descriptors, EAR files produced by the Application Server Toolkit can also include additional WebSphere-specific information used when deploying applications to WebSphere environments. This supplemental information is stored in files called ibm-xxx-xxx-xxx.xmi, also in the META-INF directory.

WebSphere Application Server V6.0 and V6.1 can also store deployment-related information (such as data sources, class loader settings, and so on) as part of an Enhanced EAR file. This information is stored in an ibmconfig subdirectory of the EAR file’s META-INF directory.

Trouble shooting errors during profile creation

The successful installation of the Network Deployment product is a two-part process:

  • The first step is using the installation wizard to install a shared set of core product files.

  • The second step is to create a deployment manager profile, an application server profile, or a custom profile.
If the installation failed during the creation of profile then check app_server_root/logs/manageprofiles/<profilename>_create.log for further information on what went wrong. The <profilename>_create.log file is an XML file that contains a record of the events that occur during the creation of the last profile.

In addition to the logs created within the core product files, the following logs are created in the app_server_root/logs/manageprofiles/profile_name directory.

  • activity.log: Compiled activity log from various installation activities

  • amjrte_config.log: Tivoli Access Manager configuration log for its Java Runtime Environment

  • collect_metadata.log: Collects metadata information about managed objects in the system to evaluate and prevent potential installation conflicts

  • createDefaultServer.log: A log from wsadmin recording the creation of the server1 process in the default profile

  • createshortcutforprofile.log:Windows tool log for creating menu entries and shortcuts

  • defaultapp_config.log:JACL script log from configuring default application resources

  • defaultapp_deploy.log:Application DefaultApplication installation log

  • node_name Service.log:Start and stop events for server1

  • filetransfer_config.log:Application filetransfer installation log

  • hamanager_config.log: Configuration log for the high availability application

  • ivt_config.log:Application ivtApp installation log

  • mejb_config.log: Application ManagementEJB installation log

  • query_config.log: Application Query installation log

  • samples_config.log: Configuration log for the PlantsByWebSphere Samples application

  • samples_install.log: Installation log for the SamplesGallery and PlantsByWebSphere Samples applications

  • scheduler.cal_config.log: Application SchedulerCalendars installation log

  • SIBDefineChains.log: Creation log for service integration bus endpoints, inbound channels and channel chains, outbound thread pool, and outbound channel and channel chains

  • SIBDeployRA.log: Deployment log for the service integration bus function

  • webui_config.log: Application administrative console installation log
The no. and logs that get created might vary based on the profile that your creating. The same set of logs get created irrespective of how your creating profile, i.e. either at the time of installation, using Profile Management tool or using the manageprofile command.

Monday, March 8, 2010

Interview questions

What is deadlock?When two threads are waiting for each other and can’t proceed until the first thread obtains a lock on the other thread or vice versa, the program is said to be in a deadlock.

What are synchronized methods and synchronized statements?Synchronized methods are methods that are used to control access to an object. A thread only executes a synchronized method after it has acquired the lock for the method's object or class. Synchronized statements are similar to synchronized methods. A synchronized statement can only be executed after a thread has acquired the lock for the object or class referenced in the synchronized statement.

What is the difference between process and thread?A thread is a separate path of execution in a program. A Process is a program in execution.

What do you understand by Synchronization?
Or
What is synchronization and why is it important?
Or
Describe synchronization in respect to multithreading?
Or
What is synchronization?
With respect to multithreading, Synchronization is a process of controlling the access of shared resources by the multiple threads in such a manner that only one thread can access a particular resource at a time. In non synchronized multithreaded application, it is possible for one thread to modify a shared object while another thread is in the process of using or updating the object's value. Synchronization prevents such type of data corruption which may otherwise lead to dirty reads and significant errors.

E.g. synchronizing a function:
public synchronized void Method1 ()
{
// method code.
}
E.g. synchronizing a block of code inside a function:
public Method2 (){
synchronized (this) {
// synchronized code here.
}
}







Java Garbage Collection

Explain Garbage collection mechanism in Java?
Garbage collection is one of the most important features of Java. The purpose of garbage collection is to identify and discard objects that are no longer needed by a program so that their resources can be reclaimed and reused. A Java object is subject to garbage collection when it becomes unreachable to the program in which it is used. Garbage collection is also called automatic memory management as JVM automatically removes the unused variables/objects (value is null) from the memory. Every class inherits finalize() method from java.lang.Object, the finalize() method is called by garbage collector when it determines no more references to the object exists. In Java, it is good idea to explicitly assign null into a variable when no more in use. In Java on calling System.gc() and Runtime.gc(), JVM tries to recycle the unused objects, but there is no guarantee when all the objects will garbage collected. Garbage collection is an automatic process and can't be forced. There is no guarantee that Garbage collection will start immediately upon request of System.gc().

What kind of thread is the Garbage collector thread?It is a daemon thread.

Can an object’s finalize() method be invoked while it is reachable?An object’s finalize() method cannot be invoked by the garbage collector while the object is still reachable. However, an object’s finalize() method may be invoked by other objects.

Does garbage collection guarantee that a program will not run out of memory?Garbage collection does not guarantee that a program will not run out of memory. It is possible for programs to use up memory resources faster than they are garbage collected. It is also possible for programs to create objects that are not subject to garbage collection.

What is the purpose of finalization?The purpose of finalization is to give an unreachable object the opportunity to perform any cleanup, before the object gets garbage collected. For example, closing an opened database Connection.

If an object is garbage collected, can it become reachable again?Once an object is garbage collected, It can no longer become reachable again.
###########################################################################

Friday, March 5, 2010

WMSG1603E - An error occurred trying to read the bundle

WMSG1603E - An error occurred trying to read the bundle


I encountered a strange error this weekend whilst installing multiple fix packs across numerous WAS systems. Despite installing these fixes numerous times, this error only occurred on one system.


I was upgrading from WAS 6.1.0.21 to 6.1.0.27 - that included WAS, SDK, IHS and Plugin fixes.


After installing the fix packs when I restarted the server I got the following error:


WMSG1603E: An internal error occurred. It was not possible to register the WebSphere MQ JMS client with the application serve

r due to exception org.osgi.framework.BundleException: An error occurred trying to read the bundle



followed by a java stack which included the following:

WMSG1603E: An internal error occurred. It was not possible to register the WebSphere MQ JMS client with the application serve

r due to exception org.osgi.framework.BundleException: An error occurred trying to read the bundle

A quick search and the reason was obvious. This WAS system does not run as root but when I checked the file permissions on the org.osgi.framework bundles in {WAS_INSTALL_DIR}/profiles/{PROFILE_NAME}/configuration the bundle in question was owned by root:


drwxr-x--- 2 wasadm wasadm 256 07 Feb 10:25 org.eclipse.update

drwxr-xr-x 4 root system 256 07 Feb 10:26 org.eclipse.osgi

drwxr-x--- 3 wasadm wasadm 256 04 Jan 11:24 org.eclipse.core.runtime


A quick change of permissions on the directory and all sub directories followed by a restart and everything came up fine.

Test connection on each node for cell scope datasource

Test connection on each node for cell scope datasource


I am sure pretty much every WAS administrator has used the "test connection" button in the WAS console to prove a JDBC datasource has been set up correctly.

Although not a problem, something that always got me and didn't seem to be as good as it could be, is the fact that if you work in a large scale enironment you may well end up setting the datasource at a cell level as this would cut down on the time it takes to set up a datasource on each node or appserver and also redude the likelihood of something being mis typed. Hoever, if you do this and then run test connection, the connection is just from the dmgr as you can see by lookiig in the dmgr logs.


So what happens if you have 10, 20 or more nodes and you want to make sure that all of them can connect correctly to the database. You could telnet from each box to the DB server on the correct port, but that just shows network connectivity rather than a full databsee connection.

I assumed this could be done in a jython script but it took me a day or 2 to figure this out. If I connected to the nodeagent in wsadmin, I couldn't get the config details of the datasources as these are accessed from a wsadmin session connected to the dmgr. But running a test connection when in a wsadmin session connected to the dmgr just does the same as the "test connection" through the admin console.

In the end, I managed to write a simple unix script, which does the following:

1. Open a wsadmin session to the dmgr to get the datasource ids and write these to a file, passing in the name of the cell

2. open a wsadmin session to each nodeagent, read the datasource id's from the file, then run a test connection.


This is the basic unix script:

###########################################################################

CELL=epwsdr21Cell

NODES="epwsdr21 epwsdr22 epwsdr23 epwsdr24 epbtdr21"

PORT=8878

echo "Running connection from each node to the datasources in WAS"

echo "Connecting to dmgr through wsadmin....."


# Connect to the dmgr through wsadmin - pass in the name of the cell - and run scropt dsconnect.py


/usr/was6/WebSphere/AppServer/bin/wsadmin.sh -lang jython -f ./jython/dsconnect.py $CELL


echo "Connecting to each nodeagent to run test connections....."


for node in $NODES;do

echo "Connecting to ${node} on port ${PORT} through wsadmin"



# Connect to each nodeagent in my list of nodes above - and run script dsconnect2.py


/usr/was6/WebSphere/AppServer/bin/wsadmin.sh -lang jython -conntype SOAP -host $node -port $PORT -f ./jython/dsconnect2.py $node


done

echo 'Complete '

#############################################################################

And here is what is in the first jython script - dsconnect.py


# Jython script to get the datasource id's once connected to the dmgr through wsadmin

import sys

print ' '

print "Getting datasources for cell " + sys.argv[0]

# First build the cell name we are interested in fromm the cell name passed from the main script

constructcell ="/Cell:" + sys.argv[0] + "/"

# get the cell id

cellid = AdminConfig.getid( constructcell )

# Get the datasource id's

print 'Datasources found are listed below:'


# In this instance I am after v4 datasources for v5 datasources use "dsid = AdminConfig.list("DataSource", cellid).splitlines()"

dsid = AdminConfig.list("WAS40DataSource", cellid).splitlines()


print dsid


# Now open a tmp file and write the list of dsid's to tfe file - this will be a string rather than a jython list


f=open('/tmp/dsconnect.out','w')

s=str(dsid)


f.write(s)

f.close()


##############################################################################

So the list of ids is written to /tmp/dsconnect.out, now the second script is called for each nodeagent I want to connect to and then run a test connection. The list that has been put into a file will be seen as a jython string rather than a list which is why I use the eval statment so it goes back into string format

# Jython script to run a test connection on a list of datasources

import sys
#

# Open temp file to get string of datasource ids and assign to a jython list

f=open('/tmp/dsconnect.out')

test=f.read()

dsids=eval(test)

f.close()


for ds in dsids:

print ' '

print 'Testing connection from ' + sys.argv[0] + ' to ' + ds

try:

outp=AdminControl.testConnection(ds)

except:

print 'Error connecting to datasource'


print outp

else:
print outp


##############################################################################