Description
Automation stitches can be created to run a CLI script and send an email message when memory exceeds specified thresholds.
This article describes how to create these automation stitches.
Scope
FortiGate.
Solution
To define Memory usage thresholds, run the following configuration:
config system global
set memory-use-threshold-extreme <percent>
set memory-use-threshold-green <percent>
set memory-use-threshold-red <percent>
end
In the above, 'memory-use-threshold-extreme' is used to define the percentage of total RAM used at which memory usage is considered extreme and new sessions are dropped.The default value is 95.
The 'memory-use-threshold-red' threshold is used to define the percentage of total RAM used at which memory usage forces the FortiGate to enter conserve mode. The default value is 88.
Lastly, 'memory-use-threshold-green' defines a percentage value of total RAM used at which memory usage forces the FortiGate to exit conserve mode. The default value is 82.
Configure the automation stitches
To create an automation stitch for high memory usage, follow the steps below.
Create an automation action to run a CLI script:
High memory usage stitch.
To create an automation stitch for high memory usage:
Create an automation action to run a CLI script:
config system automation-action
edit "high_memory_debug"
set action-type cli-script
set required enable
set script "diagnose debug cli 8
diagnose debug console timestamp enable
diagnose debug enable
diagnose debug crashlog read
get system performance status
get system session status
diagnose sys session full-stat
diagnose firewall iprope state
diagnose sys flash list
diagnose hardware sysinfo memory
diagnose hardware sysinfo slab
diagnose hardware sysinfo shm
diagnose hardware deviceinfo disk
get system arp
diagnose ip arp list
diagnose ip address list
get router info routing-table all
get router info kernel
diagnose ip rtcache list
diagnose sys top-mem 20
diagnose sys top 9 99 2"
next
end
Create an automation action to send an email:
config system automation-action
edit "auto_high_memory_email"
set action-type email
set email-to "person@fortinet.com"
set email-subject "CSF stitch alert: high_memory"
set email-body "%%results%%"
next
end
Create an automation trigger:
config system automation-trigger
edit "auto_high_memory"
set event-type low-memory
next
end
Create an automation stitch:
config system automation-stitch
edit "auto_high_memory"
set trigger "auto_high_memory"
set action "high_memory_debug" "auto_high_memory_email"
next
end
Once the memory reaches the threshold, an email will be received.
Example FGT[FGVMEVC------D] Automation Stitch:auto_high_memory is triggered.
########## script name: autod.4 ##########
========== #1, 2020-05-13 07:17:58 ==========
FortiGate-VM64 $ diagnose debug cli 8
Debug messages will be on for 11 minutes.
FortiGate-VM64 $ diagnose debug console timestamp enable
FortiGate-VM64 $ diagnose debug enable
FortiGate-VM64 $ diagnose debug crashlog read
1: 2020-05-02 06:50:56 the killed daemon is /bin/dhcpcd: status=0x0
2: 2020-05-02 06:56:59 the killed daemon is /bin/pyfcgid: status=0x0
3: 2020-05-02 08:55:01 the killed daemon is /bin/ntpd: status=0xf
4: 2020-05-02 09:01:09 the killed daemon is /bin/pyfcgid: status=0x0
5: 2020-05-02 10:01:06 the killed daemon is /bin/pyfcgid: status=0x0
6: 2020-05-02 10:12:28 the killed daemon is /bin/pyfcgid: status=0x0
7: 2020-05-02 10:23:05 the killed daemon is /bin/pyfcgid: status=0x0
8: 2020-05-02 10:26:22 Interface port1 is brought down. process_id=724, process_name="newcli".
Note: The maximum size of the CLI script action output is 16K characters as of FortiOS 6.4.0.
