FortiAnalyzer
FortiAnalyzer can receive logs and Windows host events directly from endpoints connected to EMS, and you can use FortiAnalyzer to analyze the logs and run reports.
chall_FTNT
Staff
Staff
Article Id 192847

Description

 

This article describes what is hcache and when a manual rebuild of hcache tables is advisable.

As with log viewing and searching, reporting builds upon analytical data.

But rather than running directly upon the original SQL database, report generation makes use of intermediate tables called hcache tables.

Use of hcache tables allow for faster report generation provided the hcache tables are successfully built in advance.

In most circumstances, the hcache tables will automatically be built in the background if either of the following is checked:

1) Enable Schedule.
2) Enable Auto-cache  (available in FortiAnalyzer 5.4 and later).

auto-cache acts on logs arriving (inserted) after one or both of these options has been selected.

Solution


When a manual rebuild of the hcache tables is advisabled

A manual hcache rebuild is recommended:

1) When the report grouping configuration has been changed Will speed up associated reports.
2) When a new filter is applied to report changes (without any applicable report grouping applied).
3) A dataset associated with a report is changed.

 

May in rare circumstances also be useful in the following scenario:
4) report shows 'no matching log data' but dataset shows results and chart seems to be correctly configured.
5) report generation does not complete (for issues other than high CPU utilization)

Hcache rebuild does not require a reboot, unlike SQL rebuild, and is less time-consuming than SQL rebuild.

 

Note: 

In environments with high log rates (more than 30000 logs/sec) this process can last several days.


Hcache rebuild can be run per report and so is more specific than SQL rebuild.

 

How to trigger a manual rebuild of hcache tables.
(CLI syntax below is for FortiAnalyzer 5.4.  Check the appropriate CLI reference guide for other firmware).

 

1) For a single report template:

 

# exec sql-report hcache-build <adom> <report template>

 

To check whether hcache tables have been built for each of the charts in the report:

 

# exec sql-report hcache-check <adom> <report template>

 

2. For a time period (all reports):


# diag sql hcache rebuild-report <yyyy-mm-dd hh:mm:ss>   <yyyy-mm-dd hh:mm:ss>


As an example with start and end time for one month:


# diagnose sql hcache rebuild-report "2015-08-01 00:00:00"  "2015-09-0 00:00:00"


It is possible to verify the rebuild status by examining the output of:

# diagnose test application sqlrptcached 2

Number of log table pending: all=17534(rpt=16735, fv=798, increment-fv=1)

 

If this number is stable around 0, then the rebuild has been completed.

 

The same check can be also performed with the command:

 

# diagnose sql hcache status <adom> <----- Where <adom> is the ADOM where the rebuild has been executed.

 

auto-cache order: oldest-first
....
hcache pending=17534(rpt=16735, fv=798, increment-fv=1)