Products
Webtrends Analytics 9.x
Webtrends Analytics 8.x
Issue
The Webtrends installation is displaying alerts that state the licensed page view limit is approaching or has been reached.
Resolution
Licensed page views are used during analysis of log files and every line in a log is considered a valid page view call with the exception of the following twelve file types:
gif, jpg, bmp, png, wbmp, jpeg, css, js, dtd, asmx, ascx, ico
All other file types, including files without extensions, count as page views. This count takes place prior to the application of filters, meaning that if certain filters were applied and a million page views were received, even though the reports may show little or no data, one million page views were still used and would be subtracted from the total available for that anniversary year.
Page views are based on yearly “buckets” which increment from zero on the opening of the bucket until the page view limit has been reached for that year. The page view count drops back to zero for each new anniversary year, beginning on the first of the month of the installation date. For example, if Webtrends was installed on April 15th, the the twelve-month “bucket” starts on April 1st and goes until March 31st of the following calendar year. During that twelve-month period, a maximum of 120% of page views can be analyzed. The additional 20% is allotted as a buffer for page view usage exceeding expectations. Once 120% of the page views have been used, analysis of new log file data for that period stops and will not be possible, within that period, unless a page view add-on is applied.
The criteria which specifies if log files fall within a given anniversary year is the timestamp on the entries in the log files, not the date in which it was analyzed. As an example, if page views were exhausted in the anniversary year that began on March 1, 2008 (and ran through the last day of February, 2009), and in this period the last three months of log file data (December 2008 – February 2009) had not been analyzed due to the limit being reached, on March 1, 2009, if an analysis of those last three months is attempted, the analysis will fail again because the page view limit for the previous anniversary year had already been reached and those log files fall within that reporting period. That is to say, page views are not transferrable between anniversary years. In this same example, if a log file generated on March 1, 2009 were analyzed, it would open the new yearly bucket and display report data for that day. All of the log files for the days that follow should analyze without issue because the timestamp places it within the new anniversary year. However, without purchasing additional page views, a gap will remain between the first of March and the last analysis which had occurred three months earlier.
For log files that are analyzed more than once, Webtrends will increment the page view count only the first time it is analyzed. On the first analysis, Webtrends creates a hash based on the log file’s contents in order to uniquely identify it if analyzed again. Modifying the log file’s contents will invalidate the hash, and Webtrends will consider it a new log file on the next analysis.
Note: Although the content cannot be changed, the name and location can be changed without fear of increasing the page view count.
Where did these page views come from?
Spiders and bots are one of the main reasons for the unexpected high volume usage of page views. In some cases, up to 60% of a user’s page views can be exhausted as these processes constantly crawl a customer’s web site. As stated above, filtering these from analysis will not prevent them from increasing the page view count. If spiders and bots are believed to be a problem, examination of the “Top Visitors” report will indicate whether or not this is the case. A visitor with a low number of visits, but very high number of hits is likely a spider or bot.
Link scanners, process monitors, and hardware configured to verify connectivity (such as a load balancer) can also result in high usage of page views. Some companies and applications check link server reliability, doing so by hitting the server repeatedly to see if the response states the link or hardware is down. Each of these hits to a non-excluded page file type counts as a page view.
To gain a better understanding of how page views are used, examine the “Accessed File Types” report. It does not account for Failed Hits or Cached Hits however, so it should only be used for percentage or trend info instead of overall page views. If there are very few cached hits or failed hits according to the Technical Dashboard report, this report may actually be rather close in terms of Licensed Page Views if a profile is not using filters. Assuming no filters are on the profile and there are low or no cached or failed hits, take the total from this report and subtract the file types mentioned above to get an approximate indication of the Licensed Page Views used for that time period. This number is generally far greater than most users expect because the Pages report, among others, only shows actual pages and does not include hits for pdf, swf, cgi, do, or any other file extensions, specified or not, that create lines in the log file.
To confirm the number of licensed page views used, open a command line window on the Webtrends server and navigate to the following folder:
C:\Program Files\Webtrends\common\lib
Type the following command:
wtlicman -s
This command will display the installation’s current licensing information. Relative to this article, the “MaxCollectedPV” line displays the maximum number of licensed page views which can be analyzed in an anniversary year. At the bottom of the output the “Standard pageviews collected:” and “SDC pageviews collected:” display the total number of licensed page views used for each anniversary year. For customers who use both standard web logs (W3C or Apache formats, etc.) and SDC (SmartSource Data Collector, Webtrends custom logging solution), whichever of the two is greater is what applies toward the annual limit.