dcsimg

Long query parameters are truncated, resulting in data loss

Products

Webtrends Analytics 8.x
Webtrends Analytics 9.x

Cause

Some query parameter information appears to be missing from SmartSource Data Collector (SDC) or On Demand log files. The data loss occurs on pages that send particularly long strings such as forms.

Resolution

The length limit of any URL and query stem combination in Internet Explorer is 2048 characters. If the OnPremises SDC is being used, the function in the tag that handles collection of the url/query stem is built to detect strings that are too long, and if necessary, truncate them. When truncated, the string is reduced to the first 2040 characters, followed by “WT.tu=1” at the end. WT.tu=1 serves as a marker to indicate the string has been truncated.

If WT.tu=1 appears in a log file, you may need to reconsider the site design in order to limit the amount of data being sent back to the log file. Steps can be taken to limit the length of potential answers or values, such as replacing yes/no answers with 1/0, and for using short codes in combination with translation files instead of whole-word or phrase parameter values.

In Webtrends On Demand, these limits have been increased to 16383 characters before the string needs to be truncated and will not make it into the log files to be processed.
In software versions of SDC (Smartsource Data Collector), you may choose to keep these truncated hits so they may be processed. These truncated hits will be missing any data beyond the 2048 character limit, but will make it into the log file. To log these truncated hits, change the following parameter in the dcs.cfg file from ‘false’ to ‘true’

Per the Smartsource Data Collector Configuration guide SDC Configuration

logtruncatedhits = False
to be
logtruncatedhits = True

More Information

The code is built into the collection function called dcsTag, shown below:

if (P.length>2048&&navigator.userAgent.indexOf(‘MSIE’)>=0){
P=P.substring(0,2040)+”&WT.tu=1″;
}

For OnPremises, along with the limitation of 2048 characters, there is an additional limitation of 512 characters per parameter value which the function will not detect.