Documentation Index
Fetch the complete documentation index at: https://docs.therefrain.ai/llms.txt
Use this file to discover all available pages before exploring further.
Overview
Memory operations let you collect and summarize data across multiple steps or loop iterations. Data is stored in named collections that persist throughout the execution.
Two operation types
| Type | Description | Use case |
|---|
append | Add an item to a collection | Accumulate rows, records, captured values |
aggregate | Compute a summary over a collection | Sum totals, count items, find min/max |
Append
Add a captured value to a named collection:
steps:
- ordinal: 0
description: "Extract product data"
action:
type: extract
script: "document.querySelector('.product-name').textContent"
url: "https://app.example.com/products"
riskLevel: low
requiresConfirmation: false
captures:
- name: productName
strategy: snapshot
description: "Product name"
memoryOperations:
- type: append
collection: products
source: productName
| Field | Type | Required | Description |
|---|
type | "append" | Yes | Operation type |
collection | string | Yes | Target collection name |
source | string | No | Capture variable name to append |
Aggregate
Compute a summary value from a collection:
steps:
- ordinal: 5
description: "Calculate total"
action:
type: memory
url: "https://app.example.com/products"
riskLevel: low
requiresConfirmation: false
memoryOperations:
- type: aggregate
collection: prices
operation: sum
field: price
outputVariable: totalPrice
| Field | Type | Required | Description |
|---|
type | "aggregate" | Yes | Operation type |
collection | string | Yes | Source collection name |
operation | AggregateOp | Yes | Aggregation operation |
field | string | No | Field to aggregate on |
outputVariable | string | No | Variable to store the result |
Aggregate operations
| Operation | Description | Example result |
|---|
sum | Sum of numeric values | 1500 |
count | Number of items | 25 |
avg | Average of numeric values | 60.5 |
min | Minimum value | 10 |
max | Maximum value | 200 |
concat | Concatenate string values | "A, B, C" |
unique_count | Count of unique values | 18 |
Complete example
Scrape a product listing across multiple pages and export to CSV:
title: Scrape product catalog
steps:
- ordinal: 0
description: "Navigate to product list"
action:
type: navigate
url: "https://store.example.com/products"
url: "https://store.example.com/products"
riskLevel: low
requiresConfirmation: false
- ordinal: 1
description: "Loop through products on current page"
action:
type: extract
script: "JSON.stringify([...document.querySelectorAll('.product')].map(e => e.textContent))"
url: "https://store.example.com/products"
riskLevel: low
requiresConfirmation: false
loop:
forEach: "collection:productElements"
itemVariable: product
maxIterations: 100
steps:
- ordinal: 0
description: "Capture product name"
action:
type: extract
script: "document.querySelector('.product-name').textContent"
url: "https://store.example.com/products"
riskLevel: low
requiresConfirmation: false
captures:
- name: name
strategy: snapshot
memoryOperations:
- type: append
collection: products
source: name
- ordinal: 2
description: "Count products"
action:
type: memory
url: "https://store.example.com/products"
riskLevel: low
requiresConfirmation: false
memoryOperations:
- type: aggregate
collection: products
operation: count
outputVariable: productCount
- ordinal: 3
description: "Export products to CSV"
action:
type: export
exportCollection: products
exportFormat: csv
exportPath: "./output/products.csv"
url: "https://store.example.com/products"
riskLevel: low
requiresConfirmation: false
Iterating over collections
Use collection:name syntax in a forEach loop to iterate over collected items:
loop:
forEach: "collection:products"
itemVariable: product
