Chapter 21. Optimizing Data Throughput

Table of Contents
21.1. Binary Mode Tunnel/Mirror (TCP) Connections
21.2. Tunnel/Mirror (TCP) Heartbeat and Timeout
21.3. Old Value Queuing
21.4. Un-Buffered Delivery
21.5. Screen Output
21.6. CPU Saturation
21.7. How to Optimize
21.7.1. Tunnel/Mirror (TCP) connections
21.7.2. DataHub C++ API
21.7.3. Gamma scripts

The Cogent DataHub program has a wide range of configuration options. Among these there are several settings that will optimize data throughput, which are explained in this chapter.

21.1. Binary Mode Tunnel/Mirror (TCP) Connections

TCP/IP connections to a DataHub instance can be either ASCII or binary mode. A large part of the CPU cost of transmission is marshalling messages (constructing messages at the source and parsing them at the destination). The binary mode is more efficient in both network bandwidth and CPU usage for both the sender and the receiver. Binary mode requires that the CPU architecture of the sender and the receiver agree, so you can only use this mode if you are running both the sender and the receiver on an Intel x86 CPU. The CPU gain could be as much as 50% when using binary mode. Numeric data benefits most from this option.

How to Optimize