nmon performance: A free tool to analyze AIX and Linux performance

Usage notes: This nmon tool is NOT OFFICIALLY SUPPORTED. No warrantee is given or implied, and you cannot obtain help with it from IBM. If you have a question on nmon, please go on the Performance Tools Forum site (see Resources) so that others can find and benefit from the answers. To protect your email address from junk mail, you need to create a USER ID first (takes 20 seconds at most).

The nmon tool runs on:

AIX® 4.1.5, 4.2.0 , 4.3.2, and 4.3.3 (nmon Version 9a: This version is functionally established and will not be developed further.)
AIX 5.1, 5.2, and 5.3 (nmon Version 10: This version now supports AIX 5.3 and POWER5™ processor-based machines, with SMT and shared CPU micro-partitions.)
Linux® SUSE SLES 9, Red Hat EL 3 and 4, Debian on pSeries® p5, and OpenPower™
Linux SUSE, Red Hat, and many recent distributions on x86 (Intel and AMD in 32-bit mode)
Linux SUSE and Red Hat on zSeries® or mainframe

The nmon tool is updated roughly every six months, or when new operating system releases are available. To place your name on the e-mail list for updates, contact Nigel Griffiths.

Use this tool together with nmon analyser (see Resources), which loads the nmon output file and automatically creates dozens of graphs.

Introduction

The nmon tool is designed for AIX and Linux performance specialists to use for monitoring and analyzing performance data, including:

CPU utilization
Memory use
Kernel statistics and run queue information
Disks I/O rates, transfers, and read/write ratios
Free space on file systems
Disk adapters
Network I/O rates, transfers, and read/write ratios
Paging space and paging rates
CPU and AIX specification
Top processors
IBM HTTP Web cache
User-defined disk groups
Machine details and resources
Asynchronous I/O -- AIX only
Workload Manager (WLM) -- AIX only
IBM TotalStorage® Enterprise Storage Server® (ESS) disks -- AIX only
Network File System (NFS)
Dynamic LPAR (DLPAR) changes -- only pSeries p5 and OpenPower for either AIX or Linux

Also included is a new tool to generate graphs from the nmon output and create .gif files that can be displayed on a Web site.

See the README file for more details.

Benefits of the tool

The nmon tool is helpful in presenting all the important performance tuning information on one screen and dynamically updating it. This efficient tool works on any dumb screen, telnet session, or even a dial-up line. In addition, it does not consume many CPU cycles, usually below two percent. On newer machines, CPU usage is well below one percent.

Data is displayed on the screen and updated once every two seconds, using a dumb screen. However, you can easily change this interval to a longer or shorter time period. If you stretch the window and display the data on X Windows, VNC, PuTTY, or similar, the nmon tool can output a great deal of information in one place.

The nmon tool can also capture the same data to a text file for later analysis and graphing for reports. The output is in a spreadsheet format (.csv).

Installing the tool

The tool is a stand-alone binary file (a different file for each AIX or Linux version) that you can install in five seconds, probably less if you type fast. Installation is simple:

Copy the nmonXXX.tar.Z file to the machine. If using FTP, remember to use binary mode.
Note: Version XXX replaces this example.
To uncompress the file, run uncompress nmonXX.tar.Z.
To extract the files, run tar xvf nmonXX.tar.
Read the README file.
To start the nmon tool, type nmon.
If you are the root user, you might need to type ./nmon.

Extra notes for using nmon 9 for AIX 4 only

You must be the root user or allow regular users to read the /dev/kmem file by typing the following command (as root):
```
chmod ugo+r /dev/kmem
```
If you want the disk statistics, then also run (as root):
```
chdev -l sys0 -a iostat=true 
```

How to run the tool interactively

For running the tool interactively, read the front page of the file for a few hints. Then start the tool and use the one-key commands to see the data you want. For example, to get CPU, Memory, and Disk statistics, start nmon and type:

cmd

How to get help information while running interactively

Press the h key.

Additional help information

For additonal help information, try the following:

Type the nmon -? command for brief details.
Type the nmon -h command for full details.
Read the README file.

How to capture the data to a file for later analysis and graphing

Run nmon with the -f flag. See nmon -h for the details. But as an example, try to run nmon for an hour capturing data snapshots every 30 seconds by using: Â Â Â Â

nmon -f -s 30 -c 120
nmon -fT -s 30 -c 120

The second line also captures the top processes. Both of these create the output file in the current directory called: Â Â Â Â

<hostname>_date_time.nmon

This file is in a comma-separated values (CSV) format and can be imported into a spreadsheet directly. If you are using Lotus® 1-2-3, the file needs to be sorted. (This is not required for the Excel version of the nmon analyser.) On AIX, follow this example: Â Â Â Â

sort -A mymachine_311201_1030.nmon > xxx.csv

Notes to save you time:

To load the nmon data capture file into a spreadsheet, check the spreadsheet documentation for loading CSV data files (.csv). Many spreadsheets accept this data as just one of the possible files to load or provide an import function to do this. Many spreadsheets have a fixed number of columns and rows. I suggest you collect a maximum of 300 snapshots to avoid hitting these issues.
When you are capturing data to a file, nmon disconnects from the shell to ensure that it continues running, even if you log out. This means that nmon can appear to crash, even though it's still running in the background. To see if the process is still running, type:
```
ps ?ef | grep nmon
```
Read the README file for more information about which version of nmon to run on your particular operating system.
nmon Version 10 for AIX 5 no longer uses /dev/kmem, but only public APIs. So, you don't have to chage the permissions on /dev/kmem, and there is no need to have 32- and 64-bit versions of nmon.
For AIX 5.1, 5.2, and 5.3, use nmon 10.
On AIX, don't report lslpp -Lcq bos.?p core dumps on AIX 5.1, about ML03 onwards. Also, WLM stats go missing after upgrading to AIX 5.2 ML5 to Nigel Griffiths, as these are AIX bugs. These are avoided by using nmon Version 10.
Don't use Microsoft® Windows® Telnet and use a larger window than 80 x 25 characters. Many developers use VNC and PuTTY to display nmon from a Windows machine -- why not do the same!

New features for nmon on AIX Version 10

New Features	Description
Starting up	There is also now a small shell script called `"nmon"` that starts the right `nmon` version. Place this script and `nmon`binaries in your $PATH and type: `nmon`. This version is now only compiled in 32-bit mode. So, it runs on 32- and 64-bit hardware. The idea is to make it easier to install and run.
N = NFS	NFS is completely new for `nmon` 10.
p = Partitions	This is for shared CPU partitions information -- the big p5/AIX5.3 feature.
C = CPU	This is for machines with 32 plus CPUs -- up to 128 logical CPUs by demand.
c = CPU	Details your physical CPU use -- if you are on a POWER5 with AIX 5.3 and in a shared CPU environment.
S = Subclass	This is for WLM subclasses -- by request.
a = Disk adapters	Gives you details of the disk adapter -- like their full type.
r = Resources	This includes your CPU speed in MHz.
k = Kernel	Gives some new fields.
L = Large pages	Gives you large-page stats -- popular with high-performance guys.
D = Disk	Gives you more information about your disks, disk type sizes, free, volume groups, adapter, and so forth.
n = Network	Gives you information about your network adapters details, MTU, and errors.
m = Memory	Gives you more details on where your memory is going, system (kernel) and processes, and active virtual memory.
-B	This is a start-up option to remove the boxes.

Sample output for nmon 10 for AIX 5

Figure 1 below is a sample of the screen output. It shows the opening screen for AIX 5, with lots of useful information.

Figure 1. Sample output for nmon 10 for AIX 5
Screen shot of AIX 5 opening screen

Figure 2 illustrates the details for CPU (this is a 4 CPU POWER5 machine with SMT switched on), memory use, kernel internal statistics, and disks statistics. Note: This logical partition (LPAR) is using six times its entitlement in half a CPU.

Figure 2. CPU details
Screen shot of CPU details

Figure 3 shows the details of the network, NFS statistics, and journal filesystem use.

Figure 3. Network details
Screen shot of network details

The details of the POWER5 shared processor micro-partitions statistics are shown in Figure 4 below.

Figure 4. LPAR details
Screen shot LPAR details

Figure 5 illustrates the details of the Linux version of nmon, showing the CPU (this is a 2 CPU POWER5 machine with SMT switched on), LPAR statistics, memory use, network statistics, file system use, and disks statistics. Note: The physical CPU of this LPAR is only available with SUSE SLES9 Service Pack 1 and Red Hat EL 4 Update 1.

Figure 5. Linux version of nmon
Screen shot of details of the Linux version of nmon