Component | Specifications |
CPU | 8 CPU |
Memory | 32 GB |
Storage | 500GB virtual disk running on top of Samsung SSD 360 Pro model attached to VMware ESX server |
Operating System Specifications
The
benchmark test was performed on a dedicated virtual server running
CentOS 7 Kernel 3.10.0-862 running inside the CyOPs™ virtual appliance.
Other Specifications
At the start of each test run -
The test environment contained zero alerts.
It contained only the system connectors.
The system playbooks were deactivated and there were no running playbooks.
The Playbook Execution Logs were purged.
CyOPs™ Performance Benchmarking Process
The
test was executed using an automated test bed that initiates HTTPS
calls per clock tick ( x alerts ingested per second ) which creates
alerts in CyOPs™ and then triggers a playbook for each alert created.
Details of the performance benchmarking process:
- The alerts are created by using JMeter to simulate parallel invocation of the API - ‘/api/3/alerts/’.
When an alert is created, a playbook is triggered that performs the following steps:
- Declares variables using the Set Variable Step
- Extracts
the artifacts from the source data of the alert using CyOPs Utilities
connector’s action - “CyOPs: Extract Artifacts from String”
- Updates the alert with extracted artifacts in the Closure Notes field
- Updates the Status of the alert to Closed
We iterated this test to ingest various size of workloads (for example, 25 or 50 alerts in parallel).
Finally, we performed this test for 12 hours, at the rate of 100 alerts every 20 seconds.
Observations
The
data in the following table outlines the number of alerts ingested in a
clock tick, the total time taken to ingest those alerts, and the total
time taken for all the playbooks triggered to finish execution.
Single Invocation Test
Alerts Ingested* | Total time taken to create all Alerts in CyOPs™ | Total time taken to execute all Playbooks |
25 | 3 Secs | 3.12 Secs |
50 | 5 Secs | 3.6 Secs |
100 | 7 Secs | 7.83 Secs |
*If the number of alerts ingested is 25, then the number of playbooks triggered would be 25.
Sustained Invocation Test
We continued this test to ingest 100 alerts every 20 seconds for 12 hours, and observed that 180700 alerts were created, and corresponding playbooks successfully finished executing.
