Products
Webtrends Analytics 9.x
Webtrends Analytics 8.x
Issue
After applying the default “Exclude All Spiders and Robots” filter to a profile. You continue to see traffic in the Spiders report. Why do I still see entries in the spiders report after removing “all spiders and robots” with a filter?
Resolution
The method to identify spiders is to check the user agent(browser type)on each hit and compare it against a list of known spiders. The list of user agents treated as spiders by the Spiders report and the list used by the spiders filter are stored differently within Webtrends. Occasionally, these lists may not be in sync and spiders filter will not remove all entries treated as spiders by the spiders report.
The solution to this is to create a custom browser filter specifically excluding any spiders that still appear in the report.