Products
Webtrends Analytics 8.5+
Webtrends Analytics 9.x
Cause
Scheduled and manually run events do not enter the Job Queue. There may be no indication the system is not working as expected, or an error message similar to the following may display in the status logs:
Null: Got error 134 from table handler.
Resolution
This behavior may be attributed to a number of factors affecting the scheduler process and other components to which it relates. To troubleshoot this issue, follow the steps below in sequence until it has been resolved.
Ensure the “Webtrends – Scheduler Agent” service has started.
1. Open the Windows Services snap-in and confirm the “Webtrends – Scheduler Agent” service has started. If the service was stopped, restart it.
2. In the Webtrends user interface, navigate to Scheduler > Scheduled Jobs and confirm that jobs are now processing under “Tasks currently running.”
Ensure the Webtrends Scheduler has been enabled on the primary machine and that the Scheduler agent has been enabled on all analysis engines.
1. In the Webtrends user interface’s Administration Console, navigate to Administration > System Management > Hosts and select the Scheduler link.
2. In stand-alone installations and on the primary machine of a distributed installation, ensure the options for both “Enable Scheduler agent” and “Enable Scheduler” are checked.
Note: The former option allows events to be assigned to agents on each machine on which an analysis engine is installed. The latter option enables the scheduler to direct the assignment of these tasks to each enabled agent. In a stand-alone installation the “Enable Scheduler” option should always be enabled. In a distributed installation, “Enable Scheduler” should be only be enabled on the primary machine. All other analysis engines should only have the option to “Enable Scheduler agent” checked. If “Enable Scheduler” is selected on multiple machines then each of those machines will be competing with the others for the role of master scheduler.
3. Restart all Webtrends services:
For distributed installations, stop all services on non-primary machines in the following order (not all services may be present):
“Webtrends – Scheduler Agent”
“Webtrends – User Interface”
“Webtrends – Express Analysis Engine”
“Webtrends – Express Data Mover”
“Webtrends – Data Scheduler Services”
“Webtrends – Geo Trends Service”
“Webtrends – Email Notification Service”
On a stand-alone installation, and on the primary machine of a distributed installation, stop the services in the following order (not all services may be present):
“Webtrends – Scheduler Agent”
“Webtrends – User Interface”
“Webtrends – Express Analysis Engine”
“Webtrends – Express Data Mover”
“Webtrends – Data Scheduler Services”
“Webtrends – Geo Trends Service”
“Webtrends – Email Notification Service”
“Webtrends – System Monitor”
On a stand-alone installation and on the primary machine of a distributed installation, start the services in the following order (not all services may be present):
“Webtrends – System Monitor”
“Webtrends – Email Notification Service”
“Webtrends – Geo Trends Service”
“Webtrends – Data Scheduler Services”
“Webtrends – Express Data Mover”
“Webtrends – Express Analysis Engine”
“Webtrends – User Interface”
“Webtrends – Scheduler Agent”
Start the services on all other machines (not all services may be present):
“Webtrends – Email Notification Service”
“Webtrends – Geo Trends Service”
“Webtrends – Data Scheduler Services”
“Webtrends – Express Data Mover”
“Webtrends – Express Analysis Engine”
“Webtrends – User Interface”
“Webtrends – Scheduler Agent”
4. Confirm profiles are now processing in the Job Queue.
Delete .lock files
.lock files, as the extension implies, are temporary files locked to the analysis process. These are generated when other processes on the system interfere with the analysis process. A locked process could potentially bottleneck task execution, resulting in events not moving forward in the queue until the affected process has been released. In distributed environments this may occur on individual analysis engines, the symptoms of which would be that jobs do not process on certain machines. In a stand-alone installation this will bring all events to a halt.
1. Under Scheduler > Job Queue, select “Cancel All Tasks” to remove all currently queued events from “Tasks waiting to run.”
2. In Windows Services, stop the “Webtrends – Scheduler Agent” service.
3. Navigate to the following location on the Webtrends server:
\Webtrends\modules\analysis
4. Delete any file with a .lock extension.
5. Restart the “Webtrends – Scheduler Agent” service.
6. Confirm events are now processing in the Job Queue.
Corruption of the agent.conf file
The agent.conf file is a configuration file that makes it possible for the Scheduler Agent service to interact with the other components in the installation. At times this may become corrupted for the same reason .lock files appear. A Webtrends installation keeps two copies of this file, agent.conf, the primary file, and agent.bak, a backup of the original. If the original becomes corrupted the backup can be renamed to agent.conf to restore it. If neither copy is usable, and copies are not available from a server backup, then a modified installation of the product must be performed to create a new one.
Confirm whether or not the agent.conf file has been corrupted:
1. Stop the “Webtrends – Scheduler Agent” service.
2. Navigate to the following directory.
\Webtrends\modules\agent
3. Open agent.conf with a text editor. It should display readable text, displaying configuration options and component paths. One of the first indicators of corruption will be a file size of 0 kb, however, it is possible for it to have been corrupted and still retain the original file size. When opened in a text editor it may appear blank or contain garbage characters.
4. If agent.conf has been corrupted, open agent.bak and confirm it has not been corrupted as well. If it appears usable, delete the corrupted agent.conf and rename agent.bak to agent.conf and then make a copy of the new agent.conf and rename it to agent.bak.
5. Restart the “Webtrends – Scheduler Agent” service.
6. Confirm events are now processing in the Job Queue.
Repair the MSSQL database
1. Perform the steps detailed in How do I repair a Microsoft SQL Server database using Microsoft SQL Server Management Studio?
2. Confirm events are now processing in the Job Queue.
Clear the wt_environment table in the wt_sched database
1. Stop all Webtrends services on all machines following the sequence displayed above.
2. Open SQL Server Management Studio.
3. Log into the database using Windows Authentication
4. Select “New Query” from the toolbar.
5. Type the following commands into the New Query window, then click Execute:
use wt_sched
delete from wt_environment;
6. Restart all Webtrends services in the sequence displayed above.
7. Confirm events are now processing in the Job Queue.
Disable Pipelining and Affinity
1. Perform the steps detailed inHow do I disable pipelining and affinity?
2. Confirm events are now processing in the Job Queue.
More Information
For more information on what factors can cause issues like corruption, and what can be done to prevent it, refer to How did my database or configuration files become corrupted?