Skip to main content

Capturing network requests

In addition to rendering web content and processing web pages into tabular data sets, crul can also capture the network lifecycle of loading a webpage using the requests command. This command allows us to capture performance and request/response content of third party network requests as well as a whole assortment of rich metadata for performance and security monitoring. It's also incredibly easy to use.

Example​

Full Query​

requests https://www.bbc.com/news

Stage 1: Capturing network requests​

requests https://www.bbc.com/news

This stage uses the requests command, which takes a url. It will open up the page, capture the network requests, and return them in a tabular format for processing, sorting, and export.

The requests command also takes a number flags to control several important options when accessing page, for example, the --device flag controls what kind device is used to access the page.