Skip to main content

requests

requests [url]

A summary of all external requests made from a url. NOTE: This is a slower operation as all requests are processed.

arguments:​

url​

The url for the request. (type: string)

examples:​

Query​

requests https://www.crul.com

Results after requests stage:​

request.methodrequest.urlperformance.duration...response.status
GEThttps://www.crul.com/1785/.2345...200
GEThttps://www.googletagmanager.com/gtag/js?id=G-D6YFF1FDB6201.5999...200

flags:​

--appendStage​

Used to append the results from a previous stage to the current stage. (provide a label, stage index, or boolean true to append the previous results)

--args​

array of JSON encoded values to pass to expression as input

--cache​

A boolean value of true/false that determines whether or not to use the cache. Generally most commands will default to true.

--checkpoint​

Format: "{CHECKPOINT NAME}:{COLUMN}" Used to store the value of the provided column (in the first row of results) in the provided name for use as a checkpoint in scheduled queries or other stages. Not encrypted. Can be accessed using $CHECKPOINTS.{CHECKPOINT NAME}$

--cookieStorage​

JSON serialized Cookie values to set pre-request

--device​

device to emulate, see devices command

--enrich​

Enriches each result row with the previous row. The previous columns will be appended with a _previous.

--expression​

javascript expression to evaluate after completion ES6 fat-arrow format with an argument of data passed (example "(data)=>data.map(item=>item.innerText)")

--filter​

A filter to run on the command results before completing the command. If not provided, no filter is run on the results.

--fresh​

Starts the stage as if it was a fresh query, so will not use any previous result.

--guid​

Adds a populated random guid column.

--headers​

JSON header values to set pre-request

--jquery​

inject jquery library in last object argument passed to expression (example "(data, { $ })=>$('a').html()

--labelStage​

Used to label a stage with a user provided label.

--localStorage​

JSON sessionStorage values to set pre-request

--lodash​

inject lodash library in last object argument passed to expression (example "(data, { })=>)

--mathjs​

inject mathjs library in last object argument passed to expression (example "(data, { mathjs })=>mathjs)

--maxConcurrent​

Override for the system max concurrent workers for a stage.

--proxy​

The URI for a http, https, socks(v5), socks5, socks4, or pac proxy

--randomizeHash​

Randomizes the stage hash, even if args and flags are the same.

--responseText​

include response text for non-binary resources NOTE: Due to the non-determistic size of response payloads this can have performance implications

--retry​

number attempts to sucessfully run the following task

--sessionStorage​

JSON serialized sessionStorage values to set pre-request

--stats​

Controls if a stats calculation is run on a stage after it completes.

--table​

A comma separated list of columns to include in the command results. If not provided, all columns will be included.

--type​

Each command has a default type, either "mapping" or "reducing". Some commands can operate as either, when "reducing" they will operate on all rows at once, when "mapping", they will operate on one row at a time.

--variable​

Format: "{VARIABLE NAME}:{COLUMN}" Used to store the value of the provided column (in the first row of results) in the provided name for use as a variable in other stages. Can be accessed using $VARIABLES.{VARIABLE NAME}$. Stored as an encrypted secret. Not stored across queries.

--while​

Will rerun the stage until the provided expression is valid for the first line of results.

support​

AMI_ENTERPRISE AMI_FREE AMI_PRO BINARY_ENTERPRISE BINARY_FREE BINARY_PRO DESKTOP_ENTERPRISE DESKTOP_FREE DESKTOP_PRO DOCKER_ENTERPRISE DOCKER_FREE DOCKER_PRO