This article was written by Kevin Davenport, director of IoT at Intel Corporation, and Yuta Endo,VP/GM of business development and head of APAC Operation, at Foghorn Systems. 

The Internet of Everything (IoE) is here now. Today it is estimated that manufacturers generate 2 exabytes of information daily—resulting in more “big data” than any other industry or sector. And when you consider that only 4 percent of the devices on the manufacturing floor are actually connected to a network, we can expect to see a continuing tidal wave of data coming across manufacturers’ networks as more and more “dark” devices are connected. Manufacturers are realizing that they need a new approach to properly manage the enormous amounts of data, to protect their data, and to ensure high-quality performance of applications and automation systems across their local-area and wide-area networks.

Many manufacturers have used proprietary networks in the past; however, we are now seeing a greater transition to “smart manufacturing.” A smart manufacturing environment requires a standardized IP-centric network based on Ethernet that will enable all devices within a plant to communicate to both operational and enterprise business systems. A standard Ethernet network also makes it easier to connect and collaborate with suppliers and customers to improve supply chain visibility. There are a number of distinctions between an enterprise (information technology [IT]) and operational technology (OT) network architecture that require consideration. OT networks must be designed to address radio frequency interference challenges and harsh environmental conditions, and to reliably transmit real-time deterministic safety and motion control data.

Ethernet switches properly deployed across a holistically designed IT and OT converged Ethernet architecture are required to manage the variety of IoE data, voice, and video applications. An Ethernet switch is a vital component in managing and prioritizing these applications with minimal latency (the time delay between when a message is sent and when it is received) and jitter (the variance of the latency) to meet the stringent requirements of control systems.

But what exactly is an “industrial” Ethernet switch?

Although industrial Ethernet switches use the same protocols as Ethernet applied to enterprise or office networks, industrial switches require consideration of the environment where the equipment must operate. The industrial switch must tolerate a wider range of temperature, vibration, and electrical noise than equipment installed in dedicated IT networks. Because closed-loop process control may rely on an Ethernet link, the economic costs of interruption may be high. Availability is therefore an essential criterion. Industrial Ethernet networks must interoperate with both current and legacy systems, and must also provide predictable performance and maintainability. In addition to physical compatibility and low-level transport protocols, a practical industrial Ethernet switch must also provide interoperability with higher levels of the enterprise. An industrial network must be secure from both outside intrusions and from inadvertent or unauthorized use within the plant.

The model needs to switch . . . the switch needs a model

Industrial networks often use network switches to segment a large system into logical subnetworks, divided by address, protocol, or application. Using network switches allows the networks to be broken up into many small collision domains. This reduces the risk of a faulty or misconfigured device generating excess network traffic.

The manufacturing process dictates performance requirements for manufacturing applications, work cells, machines, sensors, and actuators, as well as their geographic deployment. Understanding the performance requirements and location of these systems is critical in determining how industrial switches are deployed.

An architecture or framework is required to build a robust, flexible, and scalable design to meet the requirements of IoE.

Figure 1 illustrates the Purdue Model. This model establishes a framework for network segmentation for traffic management and policy enforcement, such as security, remote access, and quality of service (QoS). This framework leverages standard IP-based protocols including unmodified Ethernet.

The framework groups levels into the following zones for specific functions:

• Enterprise zone: Levels 4 and 5 handle IT networks, business applications/servers (e.g., email and enterprise resource planning) as well as intranet.
• Industrial demilitarized zone (IDMZ): This buffer zone is a barrier between the manufacturing and enterprise zones, but allows data and services to be shared securely. All network traffic from either side of the IDMZ terminates in the IDMZ. No traffic traverses the IDMZ. That is, no traffic travels directly between the enterprise and manufacturing zones.
• Manufacturing zone: Level 3 addresses plantwide applications (e.g., historian, asset management, and manufacturing execution systems), consisting of multiple cell/area zones.
• Cell/area zone: Levels 0, 1, and 2 manage industrial control devices (e.g., controllers, drives, I/O, and human-machine interface) and multi-disciplined control applications (e.g., drive, batch, continuous process, and discrete).

Ethernet determinism

Ensuring that a packet is sent and received in a specific time period is an important design goal for industrial networks. For the network to support predictable, real-time traffic, the design must be as simple and highly structured as possible. To control end-device latency and response time, the network has an important role to play by sending data packets consistently and predictably.

Ethernet switches normally have very low latencies, which refers to the time it takes for a network packet to travel between a source and a target. Most control operations in industrial applications can tolerate latencies of 10 to 50 milliseconds. Because control traffic frames in industrial applications are usually below 500 bytes, the latency introduced by a switch at 100 Mbps is only about 30 microseconds, with a worst-case scenario of close to 100 microseconds—well below the limit and 100 times faster than most applications require.

Selecting the appropriate industrial Ethernet switch offers manufacturers the QoS and precision time services to meet these deterministic demands.

Merging Ethernet and deterministic networks

Deterministic performance is one of the key considerations when designing a converged IT and OT architecture. Motion control applications are among the most demanding on the network from a determinism perspective. The traditional approach to handling real-time control in a motion environment is to schedule a device’s time on the network, by which all other devices are synchronized.

Some industrial network solutions, however, are based on industrial protocols like EtherNet/IP, which uses CIP Motion and CIP Sync to solve the problem of real-time motion control differently. CIP Sync uses the IEEE 1588 Standard for a Precision Clock Synchronization Protocol or Networked Measurement and Control Systems, commonly referred to as the precision time protocol (PTP), to synchronize devices to a very high degree of accuracy. CIP Sync incorporates the IEEE 1588 services that measure network transmission latencies and corrects for infrastructure delays.

The result is the ability to synchronize clocks in distributed devices and switches to within hundreds of nanoseconds of accuracy. When all the devices in a control system share a synchronized, common understanding of system time, real-time control can be accomplished by including time as part of the motion information.

Where is precision time protocol applied?

In automation technology, PTP is in demand wherever processes need to be synchronized exactly. Here, motion control is an important field of application in the broadest sense. PTP can help to synchronize drives in robots or in printing, packing, or paper processing machines, for example. High-precision clocks can also connect interactive robots, or PTP can link plant parts closely so that the processes that run can be synchronized exactly. Clocks synchronized in every component enable distributed structures to be set up and the processes to be decoupled from communicating and processing the control commands.

Benefits of IEEE 1588

IEEE 1588 is critical to making sure deterministic timing requirements are meet within a control system. The Internet of Things is putting orders of magnitude more traffic on the wire, stretching traditional control. By replacing traditional control solutions with time-based control, organizations can realize faster and higher precision goals. The IEEE 1588 standard is a solution that the industrial control industry can easily adopt to distribute precision time for time-based control on the factory floor.

The IEEE 1588 standard specifies a protocol to synchronize independent clocks running on separate nodes of a distributed control system to a high degree of accuracy and precision. The clocks communicate with each other over a communication network. In its basic form, the protocol is intended to be administration-free. The protocol generates a master-slave relationship among the clocks in the system. Within a given subnet of a network, there is a single master clock. All clocks ultimately derive their time from a clock known as the grandmaster clock. A sync message is sent periodically by any port associated with a clock claiming to be the master clock. All ports use the same algorithm, termed the best master clock algorithm. If a port of a master clock receives a sync message from a better clock, that clock ceases to claim to be a master, and the receiving port assumes the status of a slave. Likewise, if a clock with a port acting as a slave determines that it would make a better master than the current master clock, it assumes the status of master and begins to send sync messages.

Bringing dark devices to life

Manufacturers are connecting dark devices, properly managing the data, and reaping the value of more visibility into their production lines and supply chains to reduce costs and respond faster to new opportunities.

Manufacturers have been generating big data for many years. In the past, manufacturers have had a limited ability to store, analyze, and effectively use all the available data. New big data processing tools are enabling real-time data stream analysis that can provide dramatic improvements in problem solving and cost avoidance. Big data and analytics will be the foundation for areas such as forecasting, proactive maintenance, and automation. A business example of applying big data and analytics is ConAgra Mills, which makes 800 different kinds of flour for its customers. It uses predictive tools and services to forecast pricing, capacity requirements, and customer demand. This allowed the company to maximize revenues through improved margin decisions and increase production capacity utilization by 5 percent.

Standards-based IP technology, including unmodified industrial Ethernet switches, is one of the key paradigms enabling IoE. Industrial switches deployed strategically within a properly designed, converged IT and OT network architecture shine a light on previously dark devices on the manufacturing floor. Now stages are created where people, processes, data, and things integrate, synchronize, and deliver performances that drive tangible business objectives and goals. Manufacturers are capturing and leveraging these technologies to drive efficiency and innovation across their manufacturing value chains, and they are gaining a competitive edge in the market.

About the Authors
Kevin Davenport is director of IoT at Intel Corporation. Previously he was a global solutions manager for Cisco’s Industrial Intelligence.

Connect with Kevin:

Yuta Endo is VP/GM of business development and head of APAC Operation, at Foghorn Systems. He formerly was a senior manager, product management, for Cisco’s Internet of Things initiatives.

Connect with Yuta:

A version of this article also was published at InTech magazine.