Skip to main content

Common Flags

All commands in the crul query language have a set of common flags that can be used regardless of the command. These flags generally impact the way the query processor executes the command, and are quite powerful.

Below is a listing of common flags and their descriptions.

--cache​

The --cache flag is a boolean value of true/false that determines whether or not to use the cache. Generally most commands will default to true, however some commands will bypass the cache every time. For example, the timestamp command will always bypass the cache.

It is important to set --cache false when running scheduled queries on a short interval, otherwise you may continuously be reading from the cache and refreshing the cache time to live (TTL).

Generally the --cache flag is useful when you need to break the cache to fetch new results.

--labelStage​

The --labelStage flag is used to label a stage with a user provided label. This is used in conjunction with the --appendStage flag to append the results from a previous stage to the current stage.

For example, --labelStage "apiResponse".

--appendStage​

The --appendStage flag is used to append the results from a previous stage to the current stage. There are three options:

  • The first option is to set --appendStage "label" where label refers to a previously labeled stage in the pipeline. The results of the labeled stage in the query will be appended to the results of the current stage.

    Example (using a label): ... --labelStage "apiResponse" || ... || ... --appendStage "apiResponse"

  • The second option is to set --appendStage 0 or any other integer, which refers to a stage in the pipeline. 0 is the first stage, 1 the second, etc. In the example --appendStage 0, the results of the first stage in the query will be appended to the results of the current stage.

    Example (using an index): ... --labelStage "apiResponse" || ... || ... --appendStage 0

  • The third option is to set --appendStage true or equivalently just --appendStage. This option will append the results of previous stage to the results of the current stage.

    Example (appending the results of the previous stage): ... --labelStage "apiResponse" || ... || ... --appendStage

--attributes​

The --attributes flag is used to include only the columns/attributes provided in a commands result set.

The --attributes flag is useful when you have a single stage expanding into a large number of commands, which each generate a large data set. Instead of using a table command after the stage has completed and all the results have been merged together, you can instead use the --attributes flag to only return the attributes provided, which will reduce the size of the data and accelerate your query.

Example: ... --attributes "column1,column2"

--filter​

The --filter flag is used to provide a filter that will run before the command completes.

The --filter flag is useful when you have a single stage expanding into a large number of commands, which each generate a large data set. Instead of filtering after the stage has completed and all the results have been merged together, you can instead use the --filter flag to run a filter before a command in the stage completes, to reduce the size of the data and accelerate your query.

--stats​

The --stats flag is a boolean value used to control if a stats calculation is run on a stage after it completes. A stage's stats is used by the attributes exploration feature on the query page. By default, the final stage of a query will always compute stats.

You may want to set --stats false if you are not interested in a stage's stats, and want to bypass the stats calculation on a large results set.

--fresh​

The --fresh flag is a boolean value available to mapping commands that is used to create a new query pipeline mid query. Previous results will be ignored, and the stage with the --fresh flag will run as if it is the first stage of a query.

You may want to set --fresh true if you have used a --labelStage to label a previous stage, and want to start a "new" query pipeline to get a different set to then join with the labeled stage.

--checkpoint​

Format: --checkpoint "{CHECKPOINT NAME}:{COLUMN}"

Used to store the value of the provided column (in the first row of results) in the provided name for use as a checkpoint in scheduled queries or other stages. Can be accessed using $CHECKPOINTS.{CHECKPOINT NAME}$.