Custom Application Monitoring

Use ProTop's Application Monitoring feature to alert you when any business-critical application (not just OpenEdge) is down or in any state you wish to be aware of.

ProTop's application monitor is a generic interface allowing ProTop to monitor and alert for almost anything. Essentially, ProTop runs scripts defined in etc/appmon.[myApp].cfg, and generates alerts based on the output. The only requirement is that the first data output by the script must be an integer. Anything after that number is appended to the body of the alert message. The configuration details of the script that triggered the alert are also included in the message body.

For example, imagine a script erpcheck.sh that verifies if a 4gl process erpbatch.p is running:

#!/bin/bash
# Script to check if erpbatch.p is running
#
PSLine=$(ps -ef | grep _progres | grep erpbatch);RetCode=$?

if [ $RetCode = 0 ]
then
echo 0; echo $PSLine
else
echo 1; echo “erpbatch process not found” ### <-- NOTE: This message is reported by the zAppNote alert enhancer when the return code from the application script is "1", the second item in the statusList) meaning "Down"
fi

The corresponding entry in etc/appmon.myApp.cfg would be:

# id "app name" "script" "statusList"
1 "ERP Check" ./bin/erpcheck.sh "Running,Down"

StatusList is a comma-separated list of statuses, with the first entry corresponding to the "0" returned by erpcheck.sh, the 2nd entry corresponding to a "1" returned by the script, etc. The script can return as many different statuses as required by the user.

The pt3agent will assign the integer value returned by the script to the alertable metric statCode and the corresponding entry from the StatusList to the alertable metric appStatus which are passed to the alerting subsystem. So, in etc/alert.myApp.cfg, you can enter lines similar to these to control when and how you are alerted based on the values of those variables:

appStatus char <> "Running" "3:3" "hourly" "myApp &2 &3 (&1)" page
appStatus char <> "Running" "" "hourly" "myApp &2 &3 (&1)" alarm
statCode num <> 0 "" "hourly" "Appmon &2 &3 (&1)" alert

Configuration Steps

Create a resource on the portal or in etc/dblist.cfg named myApp (or any other meaningful name) with full db path = “-1” and monitored = yes.
Copy [PROTOPDIR]/etc/appmon.cfg to [PROTOPDIR]/etc/appmon.myApp.cfg
Edit etc/appmon.myApp.cfg and add the scripts you wish to run at each monitoring interval. We have only one script in our example, but you can add a line for each monitoring script you wish to run. The first column, ID, is simply a unique identifier.
Copy [PROTOPDIR]/etc/pt3agent.cfg to [PROTOPDIR]/etc/pt3agent.myApp.cfg
Comment out the existing ptInitDC line and add the following line:
```
ptInitDC "appActivity" 
```
To test, start ProTop RT and use the “e” key command to display the Application Specific Activity Monitor
```
bin/protop myApp 
```

Now, to be alerted, create a blank /etc/alert.myApp.cfg and insert alerts on metrics statCode and/or appStatus:

appStatus char <> "Running" "" "hourly" "Appmon &2 &3 (&1)" alert
statCode num <> 0 "" "hourly" "Appmon &2 &3 (&1)" alert

Since the nag frequency is hourly, you may need to restart the agent to test any changes. Simply remove tmp/pt3agent.myApp.flg, and the agent will gracefully exit. The DB Monitor will soon respawn a new agent. Otherwise, you can remove the tmp/dbmonitor.flg and run bin/dbmonitor.sh to restart both.