Products
Webtrends Analytics 8.x
Webtrends Analytics 9.x
Cause
Introduction:
Resource Control is the term given to the feature Webtrends uses to reorder non-chronological web log entries into chronological order. This feature is useful in environments where the web server’s log files create out-of-order entries due to extremely high volumes of traffic, hardware issues or complex network configurations which result in delays when logging requests as they are received on the web site. Out-of-order records may also arise from scripting errors when web logs are merged and/or scrubbed prior to analysis.
By default, Resource Control is disabled and out-of-order records are discarded. When enabled, the default threshold is ten seconds, meaning that as analysis proceeds the time stamp for each record is compared to all other records within a ten-second window and if any entries are found to be out of chronological order they will be re-sorted into the correct sequence. Records that are beyond the specified threshold are discarded.
On systems in which log file records are known to be out of chronological order, Resource Control may potentially reclaim a large number of records which would have otherwise been lost, but use of this feature incurs a significant performance hit. The degree to which this affects system resources is relative to the threshold specified, the severity by which records are out of order within this threshold and the size and quantity of web logs parsed in the analysis cycle. For example, on a high volume system on which log files split hourly and average one gigabyte in size, a resource control setting of ten seconds may potentially force the system to re-order hundreds or even thousands of records per second if records that are out of chronological order are a common occurrence.
To modify the Resource Control setting for all profiles, navigate to Web Analysis > Options > Analysis > Resource Control in the Administration console within the user interface. To make changes to an individual profile, edit the profile and select Analysis > Resource Control.
Note: Resource Control is not a user-configurable option in Webtrends On Demand.
Resource Control provides the benefit of increased report accuracy, but its use must be weighed against the potential impact it may have on system resources. It is not intended as final solution for problems with the chronology of records in web logs, but rather as a means of fine-tuning the results of analysis. For issues related to the sequence of web log records, Webtrends recommends reviewing internal processes starting with the web server(s) on through anything that may affect the log files prior to analysis. Issues with web server configuration and logging (with the exclusion of the SmartSource Data Collector) as well as web log modification are not supported by Webtrends Technical Support.