Stress Testing Nx Witness VMS with Seagate

We teamed up with Seagate Technology to stress test Nx Witness VMS with the Seagate 5u84 Storage Array using virtualization with VMWare. Below are the results.

Stress Test Participants

For over 40 years, Seagate Technology has been offering the most innovative and easily integrated storage services built on trust, affordability, and performance.

Network Optix (Nx) wants to be the engine that drives video applications – changing the way the world streams, captures, analyzes, stores, and shares video.

Goals of the Stress Test

The goal of this test was simple: to test the limits of the Seagate 5u84 high density storage array’s performance in combination with Nx Witness VMS using modern virtualization and storage technologies. 

1. Test Centralized Storage w/ Seagate 5u84

Seagate and Network Optix wanted to test the ability of the Seagate 5u84 to act as a centralized, network-based storage device for an enterprise IP video system (1000 IP cameras).

2. Deploy Nx Witness VMS in Virtual Environment

As virtualization becomes more critical for enterprise IP video applications we wanted to test the performance of an Nx Witness System running all Servers in a virtual environment with VMWare ESXi.

Testing System Setup & Architecture

The test was implemented at a partner data center in Singapore using a combination of compute, networking, and storage hardware combined with VMWare ESXi, Windows Server 2016, and Ubuntu Linux OS..

System Components

The test was performed using a combination of enterprise-class software and hardware, listed below.

Software

Nx Witness VMS
IP Video Management Software
v4.1 build 31398

VMWare ESXi
Enterprise-class Type-1 Hypervisor
v4.1 build 31398

Test Camera
Camera Emulator
v4.1 build 31195

Ubuntu Linux
Operating System
18.04 LTS / 20.04 LTS

Windows
Operating System
Server 2016

Hardware

Dell Poweredge R740 Rack Mounted Server
2 x Intel® Xeon® Gold 6132 CPU @ 2.60GHz 384GB Memory Dell 10GB NIC

Dell EMC Powerswitch 4048
Top-of-Rack Open Networking Switch
10/40GbE

Seagate 5u84
Centralized Network Storage Array

  • 84 x 8TB EXOS HDD 
  • 2) Active-Active Dual Controller Engine (4865) – Firmware GT265R009-03
  • Capable of 320,000 IOPS, 7000MBps Read, 5500MBps Write.
  • (2) Cache pairs (total 16 GB) 
  • (8) iSCSI Ports operating at 10GE SFP+ 
  • (8) Seagate 3.5” 8TB 7200k rpm NL-SAS

Storage, Network, and VMWare Configuration

Kronicles Asia implemented the Test in their Singapore office using a combination of compute, networking, and storage hardware.

Storage Configuration

Dual Storage Controller
  • Each Storage controller supported four (4) iSCSI ports
  • Eight (8) total iSCSI ports
  • Only two (2) iSCSI ports were used in each controller for test
Seagate EXOS-X Volume / Pool Setup
  • Two (2) storage pools (dg01 & dg02)
  • 42x8TB HDD
  • ADAPT data protection on Linear Class provisioning.
  • Five (5) equal size volumes were created from each storage pool
  • Each storage volume/LUN was created equally with 50 TB
  • 10 volumes matched to 10 x Nx Witness Servers with each server configured to write to individual storage volume/LUN.
Seagate EXOS-X Data Services
  • During test Data Scrubbing was turned off and no disk failures were detected to ensure optimal performance.

Network

Physical Switch
  • One (1) Dell EMC Powerswitch 4048 10GbE Switch provided core connectivity between ESXi host, infrastructure services, and Seagate EXOS-X 5u84 system.
VM Network  & Storage Configuration
  • Each ESXi host network interfaces, the core switch ports, and both the front-end and Seagate EXOS-X 10GE ports were set up with Jumbo frames.
    • MTU 9000 on the clients
    • MTU 9216 on the switches
  • Seagate EXOS-X network connectivity was provided to the Nx Witness VMs OS by VMware VMXNET3 vNICs and standard VMware vSwitches.
  • Corresponding vNICs and vmNICs were connected to a single vSwitch for front-end connectivity and a single vSwitch for back-end connectivity.
  • All XXXX physical NICs were dedicated to Seagate EXOS-X for greater redundancy and more consistent performance.
  • Each Network Optix VM was configured with a single VMXNET3 vNIC (a requirement for VMXNET3 driver). The vmNICs were configured in active-active teaming mode, and both vSwitches were set up with MTU 9000 (Jumbo frames).

Scenario 1: 1000 Full HD IP Cameras

Test Environment Configuration

Emulate 1000 Full HD (1080p) IP Cameras
  • Each Testcam VM (CCTV – CCTV10) was configured to generate 100 Full HD IP Cameras with moderate activity level.

  • A pre-recorded clip captured from a real-world IP camera was used to generate emulated camera streams.

    • Primary Stream
      Resolution: 1920 x 1080
      Frame Rate: 30 fps

    • Secondary Stream
      480 x 272
      Frame Rate: 7 fps

Use Nx Witness to record all cameras to Seagate 5u84
  • Each Nx Witness Server (Nx1 – Nx10) configured to capture 100 cameras
  • Each Nx Witness Server’s storage pointed to shared 5u84 as default recording location
  • Each Nx Witness Server (running in an Ubuntu 18.04 LTS VM) was assigned 4 vCPUs, 8 GB of RAM, and a 50GB OS Drive

Test Results

24 Hour Monitoring using Nx Witness Health
  • The results below reflect a 24 hour testing period and were gathered using:
    • Nx Witness System Health Monitoring dashboard in Nx Cloud.
    • Solarwind Network Management Software
  • Over the course of testing 1000 FHD Cameras we observed an average CPU usage (4 x vCPUs) per server of 43%, an average throughput of 735 Mbps and 0.001% dropped frames (99.999% successful frame capture).

Scenario 2: 1000 4K IP Cameras

Test Environment Configuration

Emulate 1000 4K IP Cameras
  • Each Testcam VM (CCTV – CCTV10) was configured to generate 100 4K IP Cameras with moderate activity level.

  • A pre-recorded clip captured from a real-world IP camera was used to generate emulated camera streams.

    • Primary Stream
      Resolution: 3840 x 2160
      Frame Rate: 30 fps

    • Secondary Stream
      640 x 360
      Frame Rate: 7 fps4K

Use Nx Witness to record all cameras to Seagate 5u84
  • Each Nx Witness Server (Nx1 – Nx10) configured to capture 100 cameras
  • Each Nx Witness Server’s storage pointed to shared 5u84 as default recording location
  • Each Nx Witness Server (running in an Ubuntu 20.04 LTS VM) was assigned 4 vCPUs, 8 GB of RAM, and a 50GB OS Drive

Test Results

24 Hour Monitoring using Nx Witness Health
  • The results below reflect a 24 hour testing period and were gathered using:
    • Nx Witness System Health Monitoring dashboard in Nx Cloud.
    • Solarwind Network Management Software
  • Over the course of testing 1000 4k Cameras we observed an average CPU usage (4 x vCPUs) per server of 80.5%, an average throughput of 1488 Mbps and an average of 0.005% dropped frames (99.995% successful frame capture).

Testing Conclusions

So what did we learn?

Solution Architecture
  • The test environment as configured was designed to be an atypically stressful environment, aimed at potential private cloud / enterprise applications.
  • In a real world environment compute loads would be scaled accordingly to ensure stability, total frame capture, and a level of redundancy.
  • With current architecture, however, the Seagate 5u84 was able to handle the throughput generated by both scenarios as described above with exceedingly minimal frame loss. 
Applicability to the Real World
  • Although the test was initially designed to test the ability of the Seagate 5u84 Storage Array it also was a test that was implemented and performed by Kronicles with minimal assistance from Network Optix.
  • The combination of Nx Witness VMS, VMWare, and a scalable centralized storage array from Seagate as tested would be a viable solution architecture for many potential use cases and applications, including Safe Cities, large corporate or K-12 campuses, large retail sites, stadiums and sports arenas, large industrial complexes, and any potential application that calls for virtualized VMS with a centralized storage array.

Download a PDF version of the Stress Test

Interested in sharing this with a colleague or a customer? Download the full report below.
X
X