Smart-Grid WAN communications

(Originally published Feb. 23 — Updated Oct. 26)

Introduction

For decades, power utilities have relied on dedicated telecommunications networks to connect control centers, substations, generation facilities and protection equipment. Traditionally, these networks used PDH and SONET/SDH technologies, providing predictable transmission delays, stable timing and reliable connections. The introduction of Ethernet, IP and MPLS has progressively transformed these infrastructures, enabling multiple applications to share the same communications network. SCADA, teleprotection, automation, monitoring and synchronization can now coexist over a common packet-based transport infrastructure.

However, this transformation introduces new engineering challenges.

Unlike traditional circuit-switched networks, packet networks may experience variable transmission delays, congestion, asymmetric paths and routing changes. Although these effects are generally manageable for conventional data applications, they can become critical for teleprotection and precision synchronization. Furthermore, utilities cannot simply replace all their existing equipment. Protection relays, multiplexers, serial interfaces and legacy timing systems often remain operational for decades. New WAN architectures must therefore accommodate both generations of technology.

The challenge is not simply to transport data, but to preserve the timing, performance and reliability required by critical power system applications.

Utility Communications Architecture

A power utility operates a geographically distributed communications infrastructure connecting generation plants, transmission networks, distribution substations and control centers. These networks commonly combine optical fiber, copper circuits, microwave radio and other wireless technologies.

Their principal applications include:

1. SCADA (Supervisory Control and Data Acquisition). Provides remote supervision and control of electrical infrastructure through RTUs, sensors, controllers and communications networks. Although many SCADA applications tolerate moderate communication delays, their requirements for availability, cybersecurity and predictable operation remain significant.
2. AMI (Advanced Metering Infrastructure). Enables remote meter reading, consumption monitoring and additional functions such as remote connection and disconnection. Unlike protection applications, AMI generally prioritizes scalability and communications coverage over deterministic low latency.
3. Teleprotection. Protection equipment at different locations exchanges critical information to detect and isolate electrical faults. Depending on the protection scheme, strict requirements may apply to propagation delay, delay asymmetry and communications availability.
4. Substation Automation. IEDs (Intelligent Electronic Devices) communicate using protocols such as IEC 61850 MMS, GOOSE and Sampled Values (SV). While GOOSE and SV are typically associated with substation LANs, WAN services may interconnect substations or transport selected protection and automation traffic between sites.
5. Synchronization. Clocks distributed throughout the utility provide time references for protection, measurement, disturbance recording, event correlation and monitoring. Applications such as synchrophasor measurement require precise time alignment across geographically separated locations.

These applications have different performance requirements. Consequently, a utility WAN cannot be evaluated solely by its bandwidth or Ethernet connectivity.

 

Legacy to Modern Transport Networks

From SONET/SDH to Ethernet and MPLS

Traditional telecommunications networks were designed around dedicated TDM circuits. PDH systems, including T1/E1, provided fixed-capacity channels, while SONET/SDH introduced standardized synchronous multiplexing, network protection mechanisms and improved operational management. These technologies were particularly suitable for utility applications because their transport characteristics were predictable. A circuit could provide a stable bit rate and relatively constant propagation delay without competing with unrelated traffic.

Today, Ethernet and packet-based transport technologies are widely deployed in new utility infrastructure. Their advantages include higher capacity, efficient aggregation, flexible service provisioning and the ability to transport multiple applications over shared physical resources. Ethernet provides local and carrier network connectivity. IP supports routing across interconnected networks, while MPLS and MPLS-TP can provide engineered transport services with traffic management and protection mechanisms.

However, packet transport does not automatically reproduce the deterministic behaviour of a dedicated TDM circuit. A packet network must be properly engineered to control congestion, delay variation, packet loss and recovery after failures. These considerations become particularly important when transporting legacy teleprotection services.

TDM over Packet: Preserving Investments

Despite the transition to Ethernet, many utilities continue to operate protection relays, multiplexers and controllers equipped with serial, T1/E1 or C37.94 interfaces. Replacing every device is neither technically necessary nor economically attractive, then, in many installations, these systems remain reliable and have years of useful operating life. Moreover, TDM over Packet technologies allow legacy circuit services to be transported across packet networks through circuit emulation.

This approach facilitates gradual modernization, but it introduces additional considerations. Packetization, buffering and circuit recovery mechanisms can increase end-to-end latency and create sensitivity to packet delay variation or loss. Therefore, successful circuit emulation requires verification of both the packet transport service and the recovered circuit characteristics.

Teleprotection: Beyond Network Latency

Teleprotection is among the most demanding applications carried by a power utility WAN because in high-voltage transmission systems, protection devices exchange information to identify electrical faults and isolate affected sections as rapidly and selectively as possible. Consequently, the communications channel forms part of the overall protection scheme and can therefore influence its operating performance.  Interestingly traditional teleprotection systems often use dedicated optical links, TDM multiplexers or standardized interfaces such as IEEE C37.94.

IEEE C37.94

C37.94 continues to be widely encountered in existing utility installations, and its integration into modern WANs requires particular attention to timing, alarms and transmission delays. IEEE C37.94 specifies an optical interface between teleprotection equipment and digital multiplexers, supporting N × 64 kbit/s data channels, where N ranges from 1 to 12, although the physical interface commonly operates using a 2.048 Mbit/s optical framing structure, the application data capacity depends on the number of allocated 64 kbit/s channels.

One-Way Delay (OWD) and Asymmetry

Engineers frequently measure network latency using round-trip tests. However, round-trip delay alone is insufficient to characterize many teleprotection applications. The relevant parameters include:

• OWD: transmission time from one endpoint to the other
• Asymmetry: difference between transmission delays in opposite directions
• Variation: changes in transmission delay over time
• Error: bit errors, loss of frame, signal defects and interruptions
• Recovery: changes in performance during network protection switching

Some protection schemes, particularly line differential protection, may be sensitive to communication asymmetry because measurements from remote line terminals must be appropriately time-aligned. Other schemes impose different constraints, so acceptance limits must be derived from the protection equipment and the application rather than from a universal latency threshold. A network may provide adequate throughput and low packet loss while still exhibiting excessive delay asymmetry or variable latency for a particular protection scheme.

Testing Teleprotection Circuits

A complete teleprotection commissioning procedure should verify the physical interface, bit error performance, timing recovery and end-to-end transmission characteristics. For C37.94, useful tests include BER, framing and alarm analysis, one-way and round-trip delay, frequency deviation, jitter and wander.

Where TDM circuits are transported over packet networks, these measurements should be repeated under representative traffic loads and, when practical, during network protection switching. This provides evidence that the service remains suitable for the intended protection application, rather than merely confirming that the Ethernet connection is operational.

Synchronization Across the WAN

Accurate synchronization is essential for many functions of modern power systems. Protection relays, Phasor Measurement Units (PMUs), event recorders, automation systems and network infrastructure may require time references with substantially different accuracy and availability requirements.

GNSS: the Primary Reference

Global Navigation Satellite Systems such as GPS, Galileo and other constellations provide precise time and frequency references over a wide geographical area. GNSS-disciplined clocks are widely used while the most common in substations and control centres to distribute PTP, NTP, IRIG-B, PPS and other synchronization signals. However, it is important to remark that GNSS reception can be degraded or interrupted by antenna failures, interference, jamming, spoofing or environmental conditions, for this reason, critical synchronization architectures should consider redundant references, monitoring and holdover capabilities.

A high-stability oscillator, such as a Rubidium, can maintain synchronization for a limited period after the primary reference is lost this is the hold-over. The achievable performance depends on oscillator characteristics, environmental conditions and the duration of the outage.

Precision Time Protocol (PTP)

IEEE 1588 Precision Time Protocol enables precise synchronization through the exchange of timestamped messages between clocks, with suitable hardware timestamping, network engineering and clock support, PTP can provide sub-microsecond synchronization performance. Unlike a directly connected GNSS receiver, PTP allows a timing reference to be distributed hierarchically through the network, reducing the need for every remote device to maintain its own satellite antenna. Nevertheless, PTP performance depends on the underlying network.: packet delay variation, asymmetric routes, equipment residence times and traffic congestion may affect the recovered time reference.

Boundary Clocks and Transparent Clocks can improve PTP transport by regenerating synchronization or accounting for time spent inside network equipment, according to their respective operating principles. PTP also depends on a reliable upstream reference. Transporting GNSS-derived time over PTP does not, by itself, eliminate GNSS security or availability risks.

Synchronous Ethernet (SyncE)

SyncE distributes frequency synchronization through the Ethernet physical layer. Unlike PTP, SyncE does not independently transfer absolute time or phase information. Its principal purpose is to provide a stable frequency reference to compatible network equipment, SyncE is therefore particularly useful when combined with PTP in appropriately designed synchronization architectures. For example, SyncE can stabilize the frequency of network clocks while PTP distributes the phase and time information required by remote applications.

It is important to distinguish SyncE frequency synchronization from the 50/60 Hz electrical frequency of the power grid. These are unrelated functions. SyncE deployment also requires a continuous chain of compatible network interfaces and equipment. Its performance is assessed using measurements such as frequency offset, jitter, wander, MTIE and TDEV.

Network Time Protocol (NTP)

NTP remains widely used for general-purpose synchronization in utility networks, it is suitable for network management, servers, SCADA support systems, event logging and applications where millisecond-level accuracy is sufficient.  NTP accuracy depends on network conditions, implementation and timestamping methods. It should not be regarded as equivalent to hardware-assisted PTP for applications requiring sub-microsecond alignment.

Nevertheless, NTP continues to play an important role in heterogeneous infrastructures where different systems have different synchronization requirements.

Connecting WAN to Substations

Distributing PTP across a utility WAN presents an additional challenge: different network domains may use different timing profiles and transport requirements. Telecom networks commonly employ ITU-T PTP profiles, while digital substations may use IEC/IEEE 61850-9-3 or IEEE C37.238 power utility profiles, these profiles are not necessarily interoperable through simple forwarding. They may differ in transport mechanisms, message rates, delay measurement procedures, clock behaviour and synchronization requirements. Consequently, the WAN-to-substation boundary may require a clock that terminates one timing domain and regenerates the appropriate synchronization service for the next.

A suitable synchronization gateway or Boundary Clock can also provide legacy outputs such as IRIG-B or PPS for equipment that does not support PTP, this approach allows utilities to modernize their synchronization infrastructure without replacing every installed relay, controller or measuring device. The architecture must nevertheless be engineered carefully. Profile translation, reference selection and clock regeneration should be validated against the required timing accuracy, including during reference failures. The objective is not merely PTP connectivity, but reliable synchronization across heterogeneous network domains.

 

Installation, Testing and Commissioning

The deployment of a utility WAN is not complete when devices establish connectivity. Commissioning must demonstrate that each service meets its performance and availability requirements. Different applications demand different measurements, and no single test can validate every aspect of the infrastructure.

Ethernet Service Performance

RFC 2544 provides benchmarking methods for evaluating characteristics such as throughput, latency and frame loss. ITU-T Y.1564, often referred to as Ethernet Service Activation Methodology, is designed to validate Ethernet service configuration and performance, including multiple services and their specified characteristics. RFC 6349 addresses TCP throughput testing, helping engineers identify performance limitations that may not be evident from Layer 2 measurements alone. These tests are useful for verifying transport capacity, service differentiation and network behaviour before operational traffic is introduced.

Commissioning Timing

PTP commissioning requires more than checking whether a slave clock is connected to a Grandmaster. Engineers should examine parameters including Time Error (TE), Packet Delay Variation, clock behaviour, reference changes and holdover performance. For frequency synchronization, measurements such as frequency offset, MTIE and TDEV help characterize clock stability and wander. In both cases, tests should consider the effect of traffic load, equipment failures and reference interruptions.

Teleprotection testing

Legacy services require additional physical-layer and circuit-level measurements, T1/E1 interfaces should be checked for framing, bit errors, alarms, slips, pulse shape and timing stability. C37.94 testing should include optical interface operation, BER, alarms, timing and transmission delay.

Serial interfaces may require verification of electrical signals, control circuits, clocking, data rates and error performance. Where possible, end-to-end testing should be performed with the actual service configuration.

A Practical Strategy

A structured commissioning procedure can follow four stages.

1. Physical and interface verification. Confirm optical levels, electrical characteristics, cabling, interface configuration and clock sources.
2. Transport service verification. Validate throughput, traffic prioritization, packet loss, latency and service behaviour under representative conditions.
3. Application-specific testing. Measure timing performance, teleprotection delay, legacy circuit quality and any additional requirements specified by the utility.
4. Resilience testing. Verify behaviour during loss of the primary timing source, network protection switching, link failures and restoration.

The acceptance criteria should be agreed before testing and linked to the requirements of each critical application.

ALBEDO Solutions

ALBEDO develops complementary platforms for synchronization, network access and field testing, addressing different stages of utility WAN deployment.

Net.Time Ω

Net.Time Ω is a modular synchronization platform designed to generate, distribute and translate timing references across heterogeneous infrastructures, it supports PTP, NTP, SyncE and multiple legacy timing interfaces, with configurations suitable for telecom WANs and power substations. Its capabilities include PTP profile translation, IRIG-B and PPS interfaces, redundant reference arrangements and oscillator options for improved holdover performance.

Native PRP support also facilitates deployment in redundant substation network architectures.  Net.Time Ω can therefore provide an interface between modern packet timing systems and existing equipment requiring traditional synchronization signals.

Net.Edge: Extending Packet Networks

Net.Edge addresses the integration of communications and synchronization services at network access points. The platform combines packet transport capabilities with support for legacy services and precision timing functions, including PTP Boundary Clock and Transparent Clock operation. In utility applications, this type of access platform can help extend packet-based WAN services to remote substations, renewable generation facilities and other geographically distributed sites. The integration of traditional interfaces with modern packet transport allows utilities to preserve existing assets while progressively migrating towards Ethernet-based infrastructures.

xGenius: Transport and Time

xGenius is a portable, multi-technology test platform developed for installation, commissioning and maintenance of telecommunications infrastructures. It combines Ethernet/IP testing up to 10 Gb/s with synchronization measurements and interfaces for legacy communications systems. Depending on the installed options, engineers can verify PTP, SyncE, T1/E1, C37.94, serial data communications and other transport services using a single instrument. Relevant measurements include Ethernet performance, BER, one-way delay, jitter, wander, Time Error and timing stability.

Its GNSS-disciplined internal reference and rubidium oscillator option support timing measurements requiring a stable reference, including operation during limited reference interruptions. For utility engineers, the principal advantage is the ability to investigate both transport and synchronization problems without treating them as unrelated systems.

Conclusions

The transition from traditional TDM networks to packet-based WAN infrastructures offers power utilities substantial improvements in capacity, flexibility and operational efficiency. However, successful modernization involves much more than replacing old transmission equipment with Ethernet switches and routers. Teleprotection demands predictable communication behaviour. Synchronization requires accurate and resilient time distribution. Legacy interfaces must remain operational throughout migration, while new transport networks must support increasingly demanding automation applications, these requirements make testing and commissioning an essential part of the network lifecycle.

By combining flexible synchronization platforms, packet network access technologies and comprehensive test capabilities, utilities can progressively modernize their communications infrastructure while maintaining the reliability required by critical power system operations.

Modernizing a utility WAN is not simply about transporting more data. It is about preserving the timing, performance and dependability of the services that protect and control the power grid.