Skip to main content

Internal Tokens

There are a number of tokens that are available to you in the crul query pipeline. These can be used like regular tokens to construct command arguments and flag values.

Some tokens (credentials, prompts, checkpoints, etc.) are dynamic, meaning they retrieve a preconfigured credential/prompt/etc. that must already be configured.

General purpose tokens​

$TIMESTAMP$​

The current epoch time.

$TIMESTAMP.ISO$​

The current ISO time.

$TIMESTAMP.EPOCH$​

The current epoch time.

$ROW$​

The $ROW$ internal token contains the stringified content of the row. One example of where this token can be used is with the api command's --data flag to send an entire row as the body of a POST request.

Credential tokens​

Credentials are API keys or other secrets that are used for authenticating requests.

$CREDENTIALS.{CREDENTIAL NAME}$​

You can access any credential secret value using the above token format. This is most useful with custom credentials (essentially just an key value pair where the value is encrypted). As an example, we can make an authenticated api request with a Authorization token.

api get https://www.{AUTHENTICATED-API}.com --headers '{"Authorization": "$CREDENTIALS.API_KEY$"}'

Prompt tokens​

Prompts are stored GPT prompts that can be reused in a query.

$PROMPTS.{PROMPT NAME}$​

You can access any prompt value using the above token format.

Using a prompt​

prompt "$PROMPTS.prepend$ do something $PROMPTS.append$"

Checkpoint tokens​

Checkpoints are a way to maintain and set state (key value pairs) between query stages and between queries.

An example of where checkpoints could be used across queries would be in keeping track of a timestamp in a set of results, which you could read and reset in future query executions to ensure you are only retrieving results based on the most recently set timestamp checkpoint.

$CHECKPOINTS.{CHECKPOINT NAME}$​

You can access any checkpoint value using the above token format. This is used with setting checkpoints using the --checkpoint flag.\

Setting a checkpoint​

... --checkpoint "api_next_page:result.nextPage"

Using a checkpoint​

api get https://www.{AUTHENTICATED-API}.com?nextPage=$CHECKPOINTS.api_next_page$