Skip to main content

​
Overview

Memory operations let you collect and summarize data across multiple steps or loop iterations. Data is stored in named collections that persist throughout the execution.

​
Two operation types

TypeDescriptionUse case
appendAdd an item to a collectionAccumulate rows, records, captured values
aggregateCompute a summary over a collectionSum totals, count items, find min/max

​
Append

Add a captured value to a named collection: 
steps:
  - ordinal: 0
    description: "Extract product data"
    action:
      type: extract
      script: "document.querySelector('.product-name').textContent"
    url: "https://app.example.com/products"
    riskLevel: low
    requiresConfirmation: false
    captures:
      - name: productName
        strategy: snapshot
        description: "Product name"
    memoryOperations:
      - type: append
        collection: products
        source: productName
FieldTypeRequiredDescription
type"append"YesOperation type
collectionstringYesTarget collection name
sourcestringNoCapture variable name to append

​
Aggregate

Compute a summary value from a collection: 
steps:
  - ordinal: 5
    description: "Calculate total"
    action:
      type: memory
    url: "https://app.example.com/products"
    riskLevel: low
    requiresConfirmation: false
    memoryOperations:
      - type: aggregate
        collection: prices
        operation: sum
        field: price
        outputVariable: totalPrice
FieldTypeRequiredDescription
type"aggregate"YesOperation type
collectionstringYesSource collection name
operationAggregateOpYesAggregation operation
fieldstringNoField to aggregate on
outputVariablestringNoVariable to store the result

​
Aggregate operations

OperationDescriptionExample result
sumSum of numeric values1500
countNumber of items25
avgAverage of numeric values60.5
minMinimum value10
maxMaximum value200
concatConcatenate string values"A, B, C"
unique_countCount of unique values18

​
Complete example

Scrape a product listing across multiple pages and export to CSV: 
title: Scrape product catalog
steps:
  - ordinal: 0
    description: "Navigate to product list"
    action:
      type: navigate
      url: "https://store.example.com/products"
    url: "https://store.example.com/products"
    riskLevel: low
    requiresConfirmation: false

  - ordinal: 1
    description: "Loop through products on current page"
    action:
      type: extract
      script: "JSON.stringify([...document.querySelectorAll('.product')].map(e => e.textContent))"
    url: "https://store.example.com/products"
    riskLevel: low
    requiresConfirmation: false
    loop:
      forEach: "collection:productElements"
      itemVariable: product
      maxIterations: 100
    steps:
      - ordinal: 0
        description: "Capture product name"
        action:
          type: extract
          script: "document.querySelector('.product-name').textContent"
        url: "https://store.example.com/products"
        riskLevel: low
        requiresConfirmation: false
        captures:
          - name: name
            strategy: snapshot
        memoryOperations:
          - type: append
            collection: products
            source: name

  - ordinal: 2
    description: "Count products"
    action:
      type: memory
    url: "https://store.example.com/products"
    riskLevel: low
    requiresConfirmation: false
    memoryOperations:
      - type: aggregate
        collection: products
        operation: count
        outputVariable: productCount

  - ordinal: 3
    description: "Export products to CSV"
    action:
      type: export
      exportCollection: products
      exportFormat: csv
      exportPath: "./output/products.csv"
    url: "https://store.example.com/products"
    riskLevel: low
    requiresConfirmation: false

​
Iterating over collections

Use collection:name syntax in a forEach loop to iterate over collected items: 
loop:
  forEach: "collection:products"
  itemVariable: product
See Loops & Branches for loop details.