Products
Webtrends Analytics 9.x
Webtrends Analytics 8.x
Issue
What is Smart Trimming and how does it work?
Resolution
Database driven applications such as Webtrends use tables to store information and each table of these tables is used to store unique information such as an authenticated username. During analysis, Webtrends uses an algorithm based on the age, frequency, and relevance of activity data when deciding which records to include. Smart Trimming means Webtrends collects the most significant data and aggregates records with low relevance, storing them in an “other” category.
Smart Trimming was introduced in Webtrends 7, and it allows the table to flush out older information to make room for newer unique information to be populated.
Smart Trimming has two conditions:
1. A table has to be within 10% of becoming full.
2. The smart trimming function does not remove more than 80% of a table’s content.
Example:
Authenticated Users: This table will store authenticated usernames and the amount of hits and visits each authenticated user created.
The Authenticated Users table limit by default is set to 200000. This means the table will only store the most active 200000 unique authenticated user names.
Situation A – Table will be trimmed:
Authenticated Users 1-50,000 have 5,000 page views; these 50,000 users make up 25% of 200,000
Authenticated Users 50,001-100,000 have 2,000 page views; these 50,000 users make up 25% of 200,000
Authenticated Users 100,001-200,000 have 1 page view ; these 100,000 users make up 50% of 200,000
In the above scenario, the table will be trimmed because:
a. It has 200,000 rows, and so is within 10% of being full (it is, in fact, already full)
b. The bottom 50% of the values all have the same activity of 1 page view. This is not more than 80% of the table‘s content, and so these values will be trimmed. Entries 100,001-200,000 would be trimmed out of the report.
Situation B – Table will not be trimmed:
Authenticated Users 1-5000 have 5,000 page views and above; these 5,000 users make up 2.5% of 200,000
Authenticated Users 5001-10000 have 2,000 page views and above ; these 5,000 users make up 2.5% of 200,000
Authenticated Users 10001-200,000 have 1 page view; these 190,000 users make up 95% of 200,000
In this scenario, Smart Trimming will not be invoked due to the following reasons:
a. The table is within 10% of being full (again, it is, in fact, already full) but…
b. The bottom 190,000 database detail rows for users make up 95% of the total. All would remain because this is greater than the 80% condition set by Smart Trimming.
Note: Some tables within Webtrends are unable to take advantage of Smart Trimming. The tables are:
- Pages
- Visitors
These tables cannot use Smart Trimming because other tables reference data they contain. Table limits can be adjusted under the “Resource Control” tab, found within a profile or in the “Options” menu.
More Information
For more information please see the article titled: What is the affect of table limiting sizes or memory usage in Webtrends reports?