Sunday, March 19, 2006

Monitoring Unix System Processes with Psmon

Some time ago I found a useful tool for monitoring and restarting Unix processes should they unexpectedly die. Psmon is a system monitoring script written in Perl and licensed under the Apache license that is quite useful if you run servers with critical processes on them. You can download the latest version from the psmon homepage (Version 1.39 as of this writing). Read on for tips on installing and configuring it.

There are some Perl module requirements, but the install script will try to install any modules not present. The installer is a shell script 'install.sh', located in the 'support' directory. When you run it, you'll need root privileges. Something like this should do the trick:
tar xvzf psmon-1.39.tar.gz cd psmon-1.39 sudo support/install.sh
Given that it's pure Perl, it should run on any Unix, although I've only run it on Debian and Red Hat Linux servers. If the install script doesn't work, or fails to pull in the required modules, you can install the modules manually like this (again, you'll need root privileges):
for m in Config::General Proc::ProcessTable Net::SMTP Unix::Syslog Getopt::Long; do perl -MCPAN -e"install $m";done
You can use this Perl one-liner to check definitively that all the requirements are in place, once the install is finished:
perl -e 'foreach ((Config::General, Proc::ProcessTable, Net::SMTP, Unix::Syslog, Getopt::Long)) { eval("use $_;"); print "Module $_ not present\n" if $@; }'
The psmon binary is installed in /usr/bin. Once installed, psmon can be configured through the file /etc/psmon.conf. You'll have to change both lines in the config file that read Disabled True to Disabled False before psmon will function. Once that is done, configuration is pretty simple, especially if you want psmon to just monitor processes and restart them if they die. There are a lot of other things you can do, like monitor CPU usage of a given process and restart it if it is above a certain percentage. See the /etc/psmon.conf that gets installed by default for lots of examples, or the online docs. Here is a simple config file snippet that tells psmon to monitor sshd:
<Process sshd> LogLevel LOG_CRITICAL SpawnCmd /etc/init.d/ssh restart PidFile /var/run/sshd.pid </Process>
The paths in this example are specific to Debian or Ubuntu, here is an example crafted for Red Hat's Fedora:
<Process sshd> LogLevel LOG_CRITICAL SpawnCmd /etc/init.d/sshd restart PidFile /var/run/sshd.pid </Process>
Modify the paths as appropriate for your distribution.

Once configured, you can run psmon periodically via a cron entry, like this:
*/5 * * * * /usr/bin/psmon --daemon --cron
The above will run psmon every five minutes. It will actually spawn a daemon the first time it runs, thereafter it will do nothing if it detects that it is already running, or respawn itself if needed. The '--cron' switch disables 'already running' warnings.

You can configure psmon to send email every time it has to restart a process - the config file directive is this:
AdminEmail you@yourdomain.com
This will use Net::SMTP to send mail via localhost by default - use the 'SMTPHost' directive to configure a mail relay if needed:
SMTPHost mail.yourisp.net
That's really all there is to using psmon for basic system monitoring. See the config file and docs that come with the tarball for details on other features.

Technorati Tags: , , , ,

No comments: