Some (Non-)Universal Features of Web Robot Traffic

Document Type

Article

Publication Date

5-24-2018

Identifier/URL

39720060 (Pure); 85048538946 (Scopus); 12226a81-dc8b-409e-adf9-6e4ee4d9648b (researchoutputwizard); 86a2bcd3-315d-4147-bb5e-360000d23a11 (collected_work_id); 85048538946 (QABO)

Abstract

Understanding the qualities of Web robot traffic is essential to build mechanisms for mitigating the impact of their traffic on Web systems. This paper presents an updated characterization of the navigational and session patterns of Web robot traffic across three Web servers in the United States, Europe, and the Middle East under 30 different features. The results indicate that some features may be fitted to the same heavy-tailed model across the Web servers, but the best fitting models for other features depend on the Web server. Due to some different tasks of Web robots and security policies set by website administrators, there are thus some features of Web robot traffic that cannot be universally modeled.

Comments

Understanding the qualities of Web robot traffic is essential to build mechanisms for mitigating the impact of their traffic on Web systems. This paper pre

DOI

10.1109/CISS.2018.8362266

Link to Full Text


Share

COinS