Database Log Archiving and Filtering

ProTop includes two scripts for managing Progress database log files.

NOTE: This feature is primarily for OpenEdge installations prior to OpenEdge version 11.7. Beginning in OE11.7 OpenEdge provides native database log management through the promon utility.

CAVEAT: There is a regression in the early release of this feature so be sure you are on the latest 11.7 release. 

OpenEdge database log rotator/archiver (*nix only)

lgrot.sh - archives the current database log file and truncates it.

Usage: bin/lgrot.sh  friendlyName  (from etc/dblist.cfg)

Components:

• bin/localenv - with LGARCDIR uncommented and set to your log archive directory
• bin/lgrot.sh
• etc/scheduler.*.cfg - with the line containing lgrot.sh uncommented

Here is the example entry in etc/schedule.cfg of lgrot.sh:

1 0 * * 0 lgrot.sh [friendlyName] > ${PTTMP}/lgrot_[friendlyName].err 2>&1


   or where [friendlyName] = myProdDB:


1 0 * * 0 lgrot.sh myProdDB >${PTTMP}/lgrot_myProdDB.err 2>&1

To make this work:

1. Edit bin/localenv. If it does not exist, copy bin/localenv.x to bin/localenv. Uncomment LGARCDIR and set it to the explicit pathname of your log file archive directory. Save and exit.
2. Edit etc/schedule.[custid].cfg.  If it does not exist, copy it from etc/schedule.cfg. Uncomment the line containing lgrot.sh and edit it so it appears like the example above, substituting in your database friendlyName, save, and exit.
3. Restart ProTop by removing tmp/*.flg.  ProTop will restart shortly.

Now lgrot.sh will run for the database friendlyName you have defined at 1 minute past midnight every Sunday. It copies the current database log file (and .lic file) to your database log archive directory. It is named for the unique ProTop friendly-name of the database in your dblist.cfg so if you have more than one database on the same server the archives will still be unique. It is appended with the week number of the year and then truncates (prolog) the actual database log file.

NOTE: Create a new lgrot.sh entry in your schedule.[custid].cfg for each database you want the log truncated for. Be sure the name of the .err file includes the database's friendly name. This way, the log rotation for another database will not overwrite it.

NOTE: This script overwrites the weekly archives on a yearly basis.  If you want more granular control over log file archive retention, use bin/logrotate.sh, described next.

OpenEdge weekly database log filter and rotator/archiver (*nix and Windows)

logrotate.sh[.bat] is similar to lgrot.sh above, it archives OpenEdge database log files on a strictly weekly basis and lets you define how far back in weeks you want to keep your archives. Optionally, it creates a second archive version that does not contain certain log entries. This allows you to filter out common, frequent log entries, making it easier for anomalies to stand out.

Usage WIN: bin\logrotate.bat [\explicit\path\to\db] [friendlyName]    (Yes, both!)

Usage *nix: bin/logrotate.sh [friendlyName] or foreachdb.sh logrotate.sh 

WARNING: Do NOT RUN THIS SCRIPT MORE THAN ONCE IN A WEEK! A new archive for the same week number will overwrite the previous archive for the same week, effectively erasing it.

Components:

• etc/logrotate.filter - A file containing one string per line. Log entries containing those strings will be ignored in the filtered version of the archive. If the filter file does not exist, the filtered version of the archive will not be created
• bin/logrotate.sh[.bat]
• bin/localenv[.bat] - with variables below uncommented, set and exported
• etc/schedule.*.cfg - add an entry to run logrotate.sh

To make this work:

1. [Optional] Edit etc/logrotate.filter, including one string per line that you want to be ignored in the filtered version of the log file. For example:

   (452)
   (453)
   (7129)

   The above strings will filter out the log lines that contain: (7129) or "set name", (452) or "login," and (453) or "logout."  You can use the words themselves, but the message number in parens is more exact and less likely to filter out unexpected entries. 
   
   NOTE: Ensure no blank line is at the end of your filter file.

2. Edit bin/localenv[.bat] to search for and set the following variables. If it does not exist, copy bin/localenv.x[.batx] to bin/localenv[.bat].
   1. If created above, uncomment, set, and export LOGFILTER to the explicit pathname of your Log Filter file. For example:

      export LOGFILTER=/opt/protop/etc/logrotate.filter

   2. Uncomment, set and export LOGHIST to the number of weeks of full logs to keep.
   3. Uncomment, set and export FLOGHIST to the number of weeks of filtered logs to keep.
   4. Uncomment, set and export LGARCDIR to the explicit pathname of your log file archive directory.
      
      NOTE: On Windows, remove anything on the line following your variable definitions. 
3. Add logrotate.sh to your etc/scheduler.[*].cfg
   For example:

    WIN: 1 0 * * 0 cmd /c logrotate.bat [path to db] [friendlyName] > %PTTMP%/logrotate.[friendlyName].err 2>&1
   
   *nix: 1 0 * * 0 logrotate.sh [friendlyName] > ${PTTMP}/logrotate.[friendlyName].err 2>&1
       
   or
   
   1 0 * * 0 foreachdb.sh logrotate.sh > ${PTTMP}/foreachdb.logrotate.err 2>&1

4. Restart ProTop by removing tmp/*.flg.  ProTop will restart shortly.

The logrotate.sh[.bat] will be run at 1 minute past midnight every Sunday. It will copy the current database log file to your database log archive directory, name it for the friendly name, and append the week number of the year. It will remove archives older than the number of weeks set in LOGHIST and FLOGHIST above, truncate (prolog) the current database log file, and assuming the LOGFILTER variable is set and the filter file exists, a second archive will be created minus the log entries that contain the strings in your etc/logrotate.filter file.