Products
Webtrends Analytics 9.x
Webtrends Analytics 8.x
Cause
There are a few processes and/or applications that can corrupt the analysis database for a profile, the MySQL database (if applicable), or configuration files. All of them involve interference with Webtrends’ ability to read and write these files during the analysis, when the services check the database, or when the configuration files are rewritten due to a setting change or internal process.
Resolution
Following are the most common causes of this kind of corruption, and what steps need to be taken to prevent them from being an issue:
Virus scanning software
Webtrends performs a large number of file reads and writes during the analysis process. Live virus scanners detect these changes in the file system, and will often place a read/write lock on any new or modified files in order to scan them. A virus scanner attempting to scan a file used by Webtrends during analysis can easily cause a read or write failure, resulting in incorrect data within the file and corruption of database integrity. This activity can also affect the MySQL tables as they are changed, or any other configuration file that is modified by Webtrends.
Corruption can be avoided by excluding the Webtrends installation directory structure from live scanning. If the storage locations have been changed for the Analysis Data Repository, Report Data Repository, Configuration Repository, Backup Repository, or Visitor History Export Repository, those folders need to be excluded as well. It is a good idea to exclude the web log files directory from scanning if they are on the local Webtrends system as well, but this is typically not as critical. If virus scanning is required on these directories, it needs to be done during the full scan process, and all of the Webtrends services need to be stopped while this process completes. A full scan during a time period when there is no analysis running would be safer than live scanning or a full scan during an analysis, but is still no guarantee that an analysis database will not get corrupted. If an analysis is inadvertently started during the full scan, the likelihood of a corrupted database increases exponentially. The only way to ensure this does not occur is to have all of the Webtrends services stopped during the scan.
Windows Indexing service
This service is used to index files and folders for faster searching in Windows. As this service needs to read the file name as well as the contents, the same type of corruption resulting from live virus scanning can occur during indexing if the service attempts to access files at the same time Webtrends is reading or writing from them. The only option for the Indexing service is to disable it.
Backup software
As with virus scanning and indexing, backup software typically needs to put a lock on files so they cannot be written to while the file is being archived. Most backup software applications attempt to be as non-invasive as possible. Even so, with the amount of reads and writes being done by Webtrends, especially during the analysis process, there is no way to ensure that some sort of conflict will not occur therefore it is strongly recommended that the Webtrends services should be stopped before the backup begins. The “Webtrends – Scheduler Agent” service constantly makes calls to the database to determine if there are any events that need to be run. When these calls are made, there are tables within the database that obtain an exclusive lock. If the backup software attempts to copy one of these table files during the time it is locked, corruption will likely occur. NOTE: For more information on handling the backup of Webtrends’ files and folders please contact technical support. At the very least, there should be no analysis running at backup time, but there is still the strong possibility for corruption if the services are not completely shut down.
Script blockers
While not nearly as dangerous to the integrity of the database, a script blocker can still cause its own set of problems. At the very least, it can cause the analysis process to fail. Removing the script blocker would be preferable, but making sure that the scripts originating in the Webtrends directory structure are considered safe by the script blocking software will insure that the analysis process is not interrupted.
Pop-up blockers
While this does not cause corruption to the database, it is related to the subject. Pop-up blockers can prevent some Webtrends processes from running because some pop-ups are used for verification. Pop-up blockers will also prevent access to the help files.
Slow hard drives
Webtrends is a very resource intensive application and will use up 100% of a CPU during the analysis process, can fill up 1GB or more of physical RAM, and needs to perform a massive amount of reads and writes to the systems hard drive(s). Slow IDE hard drives, or older and slower SCSI drives, will simply not be able to keep up with the activity Webtrends generates. When this happens, read/write errors are almost certain to occur. For installations that are analyzing small amounts of data (1MB to 20MB worth of daily log files), a 10,000 RPM hard drive should be the minimum. For installations analyzing larger amounts of data (20MB to 500MB worth of daily log files), the system should have 15,000 RPM SCSI hard drives (preferably in a RAID 5 configuration) or 10,000 RPM SATA drives. For installations analyzing even more data on a daily basis, high end SCSI drives in a RAID 5 configuration are a requirement. Hard drive space can be a direct cause of corruption as well. This is especially true if the log files being analyzed are compressed or accessed via FTP. Even with 500GB of free hard drive space, if the log files for the analysis are compressed or accessed via FTP, this may not be enough. For an explanation on this and how to resolve the issue, see Knowledge Base article 052133.
Too many Custom Reports
Each Custom Report that is added to a profile also adds a new level of complexity to the analysis process. Due to the nature of custom reports, each one requires its own table or set of tables. As these are updated during the same time that the standard analysis tables are updated, the additional tables mean additional hard drive activity. As the previous example pointed out, that additional activity raises the probability of corruption in the analysis database files. Webtrends recommends using no more than 20 custom reports for a single profile. If additional custom reports are required, consider creating a copy of the profile and adding the custom reports to the copy.
Not enough memory
If Windows runs out of physical RAM to store information for currently running processes, it starts swapping that data out to the hard drives. This will not only have a huge impact on the speed and performance of a system, it will provide even more chance for read/write errors while Webtrends is manipulating data on the system’s hard drives. While CPU power is extremely important when it comes to performance, skimping on RAM could be even more detrimental. Not enough memory, especially when running GeoTrends, will almost certainly cause database corruption at some point.
VMware
Webtrends does not support VMware or other virtual environments for use with Webtrends software implementations for versions earlier than Webtrends 8.5x. This is due to known conflicts between the function of virtual environments and the resource requirements of certain Webtrends components that can result in corruption in Webtrends’ data and configuration files. With the release of Analytics 8.7d, Webtrends began supporting VMware ESX and ESXi servers with the provision that the RAM be configured as Reserved (as opposed to Limit).
Firewall software
Depending on the type of the firewall, it may try to block one part of an analysis process while another continues to work. While rare, this still has the potential for causing corruption. And like script blocking, it can cause an analysis to fail. There should not be a local firewall running on the Webtrends server.
Once corruption occurs in a profile’s analysis database, there is no method available to repair that corruption. There is a data integrity check that can be run, but that is all that can be done, and it only verifies whether or not the database is corrupt. The only options are to restore an archive/backup (assuming the profile or installation is being backed up), or to clear the analysis data and re-analyze the profile.