Ever wondered which files have been changed on your system?

The "ff" command can scan the inodes of a filesystem or logical volume and produce a status report, for instance to see all the files that have changed within the last 24 hours in the root filesystem:

# ff -m -1 -u /dev/hd4
ff: /dev/rhd4: 43 files were selected
./etc   12          root
./etc/objrepos/CDiagAtt 97          root
./etc/objrepos/CDiagAtt.vc      98          root
./etc/objrepos/CDiagDev 99          root
./etc/objrepos/CuAt     101         root
./etc/objrepos/CuAt.vc  102         root
./etc/objrepos/SRCnotify        120         root
./etc/objrepos/SRCsubsvr        121         root
./etc/objrepos/SRCsubsys        122         root
./etc/objrepos/SWservAt 123         root
...
..



To list files that have not been changed for more than 30 days:

# ff -a +30 /dev/hd4
./lpp/bos.net/deinstl/bos.net.tcp.server/6.1.8.15/bos.net.tcp.server.prvcmd.secpreapply        77572       root
./lpp/bos.net/deinstl/bos.net.tcp.server/6.1.8.15/bos.net.tcp.server.lib.sec.prapply   77573       root
./lpp/bos/bos.rte.tty/6.1.8.15  77600       root
./lpp/bos/deinstl/bos.rte.tty/6.1.8.15/bos.rte.tty.sec.preapply 77601       roo
./lpp/bos/deinstl/bos.rte.tty/6.1.8.15/bos.rte.tty.prvcmd.sec.preapply  77602      root
./lpp/bos/bos.rte.archive/6.1.8.15      77632       root
./lpp/bos/bos.rte.archive/6.1.8.15/bos.rte.archive.rl   77633       root
./lpp/bos/bos.rte.archive/6.1.8.15/bos.rte.archive.inventory.restore    77634      root
...
.

To list the paths corresponding to i-node numbers 21016 and 8216,enter:

# ff -l -i 21016,8216 /dev/hd3
ff: /dev/rhd3: 2 files were selected
ff: /dev/rhd3: 0 link names were detected
./.workdir.4587694.6488228_1    8216
./.workdir.16056422.15925484_1  21016

Preparing for PCI-DSSv3

Anyone who (indirectly) handles card-holder data must be compliant by the end of this year, and have procedures in place that they can demonstrate to an external auditor or PCI assesor, or they risk large fines and penalties.

The main points are:

1. You must install and maintain a firewall(s) that protect all cardholder data and prevent any direct inbound connections to your production network.
2. All vendor-supplied/default passwords must be (regularly) changed and a procedure put in place to audity and control this activity.
3. All cardholder data must be encrypted and protected at rest.
4. All cardholder data must be transmitted in encrypted form particularly when it is uses public or shared networks.
5. All systems must be protected against malware and viruses, and the protection software must be regularly updated
6. All systems and applications must be hardened in order to limit access
7. All access to cardholder data should be restricted to those that actually need it for their job and that information should be limited to enable them to carry-out that function
8. Strict access-control and authentication must be in place in order to limit and record all (attempted) access to system components or applications
9. All physical access to cardholder data must be restricted.
10. All (attempted) accessto cardholder data or systems must be recorded.
11. System and processes should be regularly tested to ensure continuous compliance.
12. There must be an adequate security policy in place which covers both physical components and the people using them.

These rules affect you whether you have local hardware or rely on a third-party to provide resources in the cloud.

I have spent the past four years creating an AIX scanning solution which tests more than 1000 aspects of your AIX system build and configugration, and produces a detailed HTML report which can be used to quickly audit your systems and to identify ommisions, misconfigurations, and mistakes.

/proc filesystem

There is a lot of useful information about all the active processes in "/proc" filesystem, unfortunately you need canot just "cat" the files as "/proc" is not a normal filesystem and contains a snapshot of the current system status, and may well have changed before you get the chance to examine it.  e.g.

# ps
      PID    TTY  TIME CMD
  8847424  pts/0  0:00 -ksh
  9437218  pts/0  0:00 ps

# find /proc/8847424
/proc/8847424
/proc/8847424/as
/proc/8847424/cred
/proc/8847424/ctl
/proc/8847424/cwd
/proc/8847424/fd
/proc/8847424/fd/0
/proc/8847424/fd/1
/proc/8847424/fd/2
/proc/8847424/fd/10
/proc/8847424/fd/63
/proc/8847424/lwp
/proc/8847424/lwp/38404245
/proc/8847424/lwp/38404245/lwpctl
/proc/8847424/lwp/38404245/lwpsinfo
/proc/8847424/lwp/38404245/lwpstatus
/proc/8847424/map
/proc/8847424/object
/proc/8847424/object/a.out
/proc/8847424/object/jfs2.10.5.487607
/proc/8847424/object/jfs2.10.5.1664
/proc/8847424/object/jfs2.10.5.4220
/proc/8847424/object/jfs2.10.5.13168
/proc/8847424/object/jfs2.10.5.339
/proc/8847424/object/jfs2.10.5.4250
/proc/8847424/psinfo
/proc/8847424/sigact
/proc/8847424/status
/proc/8847424/sysent
/proc/8847424/root
/proc/8847424/mmap

Just listing the "/proc" filesystem tree for a process does give you a good idea of what the system is doing and which resources it requires, however you need to use specialist tools such as:

Tool	Description
procstack	Get Process Stack Trace
procflags	Show Pending and Held Signals for Process
procsig	Display Signal Action and Handlers for Process
procfiles -n pid	Report stat and fcntl Info for All Open Files in Each Process
procwdx	Display the Current Working Directory of the Process
proctree	Display the Process Tree

to get more information.

The "/proc" entries contain the following information:

Directory or filename	Description
/proc/pid/as	Address space used by this process.
/proc pid/cred	Contains a description of the credentials associated with this process.
/proc/pid/ctl	Process control file.
/proc/pid/cwd	A link that provides access to the current working directory of this process. Any process can access the current working directory of the process through this ink, provided it has the necessary permissions. If you run "strings" against this file it gives the same output as "ls".
/proc/pid/fd	Contains files for all open file descriptors for this process.
/proc/pid/map	Address space map for this process.
/proc/pid/object	Directory for objects.
/proc/pid/psinfo	Process status information.
/proc/pid/sigact	Signal actions for this process.
/proc/pid/status	Process status.
/proc/pid/sysent	System call information for process PID.
Thread specific files
/proc/pid/lwp/tid	Directory for thread.
/proc/pid/lwp/tid/lwpctl	Control file for thread.
/proc/pid/lwp/tid/lwpsinfo	Process status info for thread.
/proc/pid/lwp/tid/lwpstatus	Status of thread.

Creating your own LPPs

It is common practice to create packages in RPM format, however very few people also realise that you can also create your own native AIX-format LPP (Licenced Program Product) packages with very little effort.

Start by downloading and installing the "freeware.bull.mklpp.rte" package from http://www.bullfreeware.com/

The software is installed in "/usr/local/lib/mklpp-1.2" and you will find a "README" file and example directory that will get you started. The software itself is very old, however it should work for any version of AIX.

Next create a dedicated filesystem or directory to hold your LPP build tree. e.g. "/lppbuild" and cd to the new location.

Create a new LPP name and version number e.g.

# cd/lppbuild
# newlpp mylpp-1.1.2 gnu.mylpp
creating LPP structure for Usr-only LPP

This creates the following files and directories:

./mylpp-1.1.2
./mylpp-1.1.2/.info
./mylpp-1.1.2/.info/freeware.gnu.mylpp.rte.post_i
./mylpp-1.1.2/.info/freeware.gnu.mylpp.rte.pre_i
./mylpp-1.1.2/.info/freeware.gnu.mylpp.rte.unpost_i
./mylpp-1.1.2/lpp_name
./mylpp-1.1.2/usr
./mylpp-1.1.2/usr/local
./mylpp-1.1.2/usr/local/bin
./mylpp-1.1.2/usr/local/lib
./mylpp-1.1.2/usr/local/lib/mylpp-1.1.2
./mylpp-1.1.2/usr/local/man
./mylpp-1.1.2/usr/lpp
./mylpp-1.1.2/usr/lpp/freeware.gnu.mylpp

Manually create a text file e.g. "cat ./.info/freeware.gnu.mylpp.rte.copyright" and include any legal information that should be distributed with the package.

Next edit the three scripts in ".info" so they contain the correct list of pre and post-reqs, and any package dependencies.

Unpack your new package files immediately beneath this directory and update the "./lpp_name" to include a list that will be included in the package e.g.

4 R I mylpp.gnu.mylpp {
freeware.gnu.mylpp.rte 01.02.0004.0000 01 N U en_US Example LPP
[
%
/usr/local 9
/usr/local/bin 229
/usr/local/lib 3
/usr/local/lib/gzip-1.2.4 146
/usr/local/lib/gzip-1.2.4/sample 24
/usr/local/man 3
/usr/local/man/man1 55
/usr/local/info 71
INSTWORK 19 19
%
%
%
%
]
}

Change to the package subdirectory and generate the LPP:

# cd mylpp-1.1.2/
# mklpp
rm -f .info/freeware.gnu.mylpp.rte.al
rm -f .info/freeware.gnu.mylpp.rte.size
rm -f .info/freeware.gnu.mylpp.rte.inventory
rm -f .info/backup_files
rm -f usr/lpp/freeware.gnu.mylpp/liblpp.a
rm -f usr/lpp/freeware.gnu.mylpp/inst_root/liblpp.a
rm -f .info/liblpp.a
rm -f /home/root/lppdir/out/gnu.mylpp-0.0.0.0.bff /home/root/lppdir/out/gnu.mylpp-0.0.0.0.bff.asc /home/root/lppdir/zip/gnu.mylpp-0.0.0.0.exe /home/root/lppdir/bff/gnu.mylpp-0.0.0.0.bff /home/root/lppdir/bff/gnu.mylpp-0.0.0.0.bff.asc
Making directory list
...
..

You will need to experiment with this before you get your package to behave exactly as expected and care must be taken to avoid overwriting any files that belong to another package.

AIX 7.1 Introduces NTP version 4

AIX 7.1 now includes support for NTP version 4 which is far more accurate than previous versions.

To check which version is active examine the symbolic link:

If you are using version 3 you should see:

# ls -l /usr/sbin/xntpd
lrwxrwxrwx    1 root     system           20 Apr 11 11:49 /usr/sbin/xntpd -> /usr/sbin/ntp3/xntpd

If you are using version 4 you should see:

# ls -l /usr/sbin/xntpd
lrwxrwxrwx    1 root     system           20 Apr 11 11:49 /usr/sbin/xntpd -> /usr/sbin/ntp4/xntpd

Eac version of NTP is meant to be backwards compatible however you should always do extensive testing before relying on this in production.

Firmware Assisted Dumps

POWER6® processor-based systems enable system dumps to be firmware assisted. When performing a firmware-assisted dump, system memory is frozen and the partition rebooted, which allows a new instance of the operating system to complete the dump.

Firmware-assisted dump is now the default dump type in AIX V7.1, when  the hardware platform supports it. The traditional dump remains the default dump type for AIX V6.1, even when the hardware platform supports firmware-assisted dumps.

To see which kind of dump you are using:

# sysdumpdev -l
primary              /dev/lg_dumplv
secondary            /dev/sysdumpnull
copy directory       /var/adm/ras
forced copy flag     TRUE
always allow dump    FALSE
dump compression     ON
type of dump         traditional

To enable firmware assisted dumps:

# sysdumpdev [ -t { traditional | fw-assisted } ] [ -f {disallow, allow, require }]

Full memory dump options available with the sysdumpdev -f command

Option Description

disallow                         Selective memory dump only. A full memory system dump is not allowed. This is the default.

allow | allow_full          The full memory system dump mode is allowed but is performed only when the operating system cannot properly handle the dump request.

require | require_full     The full memory system dump mode is allowed and is always performed

AIX Version 6.1 Technology Level 1 introduced support for an iSCSI device to be configured as a dump device for firmware-assisted system dump. The sysdumpdev command supports configuring an iSCSI logical volume as a dump device. In AIX V6.1, it was mandatory that this dump device be located on an iSCSI boot device.

AIX V7.1, firmware-assisted dump also supports dump devices located on arbitrary non-boot iSCSI disks. This allows diskless servers to dump to remote iSCSI disks using firmware-assisted dump. The iSCSI disks must be members of the root volume group.

Does your system use Solid State disks

Some newer AIX servers come with SSD (Solid State) disks in order to increase performance and energy efficiency. You need to be running at least AIX 6.1TL06 to support them.

SSD disks cannot be mixed with traditional disks and cannot share the same volume-group. The LVM commands such as mkvg have been updated to include SSD-only options e.g. mkvg -X SSD

To see if you are using SSD:

# lsdev -Cc disk | grep SSD
hdisk9 Available 01-08-00 SAS RAID 0 SSD Array
hdisk10 Available 01-08-00 SAS RAID 0 SSD Array
hdisk11 Available 01-08-00 SAS RAID 0 SSD Array