Service Telemetry in ProTop

1. Meaning of Service Telemetry

Service Telemetry is an advanced observability feature within ProTop that provides a granular, call-level breakdown of activity within Progress Application Server for OpenEdge (PASOE). While standard PASOE monitoring views the application as a single service, Service Telemetry dissects that service into individual calls, tracking the characteristics and performance of each request.

2. Benefits

• Granular Visibility: Breaks down high-level service data into individual calls, identifying exactly which requests are driving load or causing delays.
• Performance Troubleshooting: Tracks metrics like call rate and run time (minimum, average, and maximum) to pinpoint slow services or unexpected spikes.
• Transport-Specific Insights: Reports activity by transport type (e.g., APSV, REST, SOAP, and Web Handler), allowing for better architectural analysis.
• Proactive Detection: Supports identifying “bad” behaviors, such as memory leaks, before they escalate into system-wide outages.

3. Configuration

Prerequisites

• Service Telemetry License: A valid license from Consultingwerk is required to utilize the Service Telemetry feature. We are assuming you have already worked with Consultingwerk to get it installed and running.
• PASOE Instance up and running: Service Telemetry is compatible with multiple OpenEdge PASOE versions.  NOTE: It can also be used with Classic Appserver, which might be very helpful for people who have yet to migrate to PASOE.
  

Setup Steps

1. Data Collector Activation: Ensure the Service Telemetry data collector is active.
   
   Add "svcTelemetry" to ptInitDC in your pt3agent.*.cfg.  Note that because it does not require a database connection, it must be listed in the nodbDataCollector property in etc/protop.cfg.  It is there ordinarily, by default (and commented out in protop.*.cfg).  If you have overridden this variable and this data collector is not present, you will need to add it.
2. Log File Configuration: Copy etc/svctelemetry.cfg to etc/svctelemetry.<custId>.cfg (or other valid config file hierarchy name) and edit svcLogDir and svcLogTempl variables to correspond with your active log directory and log file naming convention. For example:
   
   # etc/svctelemetry.cfg
   #
   # The service telemetry log files contain New Line Delimited Json
   # describing calls to the services being monitored
   # each line of JSON log contains the following fields:
   #
   #       key             character       a unique key describing the monitored service
   #       runTime         integer       the runtime (in milliseconds) for this call to the service
   #
   # any additional fields in the JSON are ignored
   #
   # assign the following ProTop fields to the values you have established in your
   # Service Telemetry configuration
   
   svcLogDir          /usr/oe12.8/wrk/pasoe/s2k/ablapps/s2k/telemetry
   svcLogTempl     telemetry_wss-qa-l1.wss.com.pasoe128.s2k*ndjson
   
   # always end with a comment
   

4. How to Use It

• ProTop RT (Real-Time): Access the Service Telemetry Panel of ProTop RT by using the "4" command key to view live call rates and run times.  It takes a couple of monitoring intervals to render, so be patient.
• Web Portal: Service Telemetry Dashboard - View detailed historical charts and graphical breakdowns. This is particularly useful for identifying long-term trends and correlating them with other system events.

5. How to Interpret the Data

When analyzing Service Telemetry data in ProTop RT or the Portal, focus on the following key metrics :

Interpreting Status Signals

• Red Status: In ProTop dashboards, red indicates a critical alert requiring immediate attention.
• Correlation: Use Service Telemetry in conjunction with general PASOE request wait data. If request waits are high, check Service Telemetry to see if a specific, long-running call is blocking the session manager.