ProTop Knowledge Base

Why Does SYNCIO Seem To Disagree With My IO Throughput Ttest? (Or My Vendor's Promises)

It is common for a system admin to think that their disks have much better throughput than syncio. Have a look at why this might be the case.

Syncio exists precisely because vendor tools and marketing materials commonly report performance data for large streaming transfers using asynchronous operations and other vendor-favorable approaches.

This is different from what the OpenEdge database (and syncio) does.

Syncio is telling us what the OpenEdge database is experiencing when it writes data to the before-image file (aka the "undo log"). It says that when a BI file was expanded by 96MB, it took X amount of time to do so, and, therefore, the throughput was Z MB/sec.

This is an end-to-end test of the O (write) in IO from the OpenEdge database application point of view. If syncio reports values substantially lower than your vendor's test, their test is not equivalent to what OpenEdge is doing.

Test it yourself

NOTE: We'd like to encourage you to test any disk system you are considering using with the OpenEdge database, preferably before you commit to it. Ideally, you would use the syncio test as provided, but that requires OpenEdge and ProTop to be installed and may not be practical. Instead, we have provided below a simple dd test that is nearly equivalent.

If you already have OpenEdge and ProTop installed, at the top of bin/syncio.[sh|bat], there is an example of "dd" that should provide very similar results to syncio. For those of you who do not have ProTop, the command is:

dd if=/dev/zero of=./test.out bs=16k count=6144 oflag=dsync

The bits about "bs=16k" and "oflag=dsync" are CRITICAL. You are only running an apples-to-apples test if you include those.

of=./test.out # point this to the directory you are testing (db, ai and bi files); same given to syncio

bs=16k # this is blocksize syncio uses to grow the before-image file

count=6144 # number of 16k blocks to be added to be equivalent to the bigrow (syncio) test

oflag=dsync # these writes MUST be synchronous

NOTE: If you mistakenly run this test with the current directory on the "instance drive," performance will likely be excellent since that drive is internal to the machine rather than on a SAN somewhere.

Don't be fooled by "burst"

In cloud deployments, one of the frequent issues is that the cloud provider will allow "burst" performance well over the committed performance. This frequently results in tests that outperform expectations, especially if that testing is done without an actual production workload. This behavior often fools sysadmins into thinking they have "great" IO performance when they actually have mediocre performance.

Why doesn't the dd test match my syncio values?

If the "proutil bigrow" (syncio) result differs from the "dd" described above, then some potential sources of discrepancy are:

The "guarantee" from the provider wasn't a guarantee
The same filesystem was not being tested by dd and syncio
The other workload on the target filesystem was not equivalent when the comparisons were made
One of the tests took advantage of "burst mode."
The parameters given to "dd" are not equivalent to what OpenEdge is doing
The version of "dd" being used may not support the parameters required

What should my syncio speed be?

From a write-intensive OLTP database point of view, and looking at the syncio trends in the ProTop Portal:

< 10 MB / second = very bad; replace the db IO subsystem ASAP
10MB / second = barely acceptable, order storage now
20MB / second = Start looking for better storage
30MB / second = Start thinking about replacing the storage subsystem
100MB / second = excellent!

CAVEAT: On less write-intensive systems where users are not complaining of slowness, look at MTX latch waits. MTX latch requests are made when a process needs to write to disk. If MTX latch waits are low or nonexistent, make a note that your IO subsystem might be a future point of contention and evaluate it more closely as your business grows.