Skip to content

Sequence Pattern Recognition Test Manifold

Overview

The Sequence Pattern Recognition test manifold evaluates a model's ability to apply complex mathematical rule systems to generate sequence continuations. Models must process a base pattern along with multiple interference rules that create conditional modifications, then generate accurate sequence terms through systematic rule application.

Task Description

Models are presented with an initial sequence of numbers followed by a set of rules that govern sequence generation. The base rule establishes the fundamental pattern (arithmetic, geometric, etc.), while interference rules create exceptions and modifications based on various conditions. After understanding all rules, the model must generate the next specified number of terms in the sequence.

Key Features: - Multi-Layered Rule Systems: Processing base patterns with interference rule modifications - Conditional Logic: Handling rules that apply only under specific mathematical conditions - Rule Composition: Understanding how multiple rules interact and override each other - Mathematical Operations: Working with arithmetic, geometric, and custom mathematical functions - Sequential Processing: Generating accurate sequences through systematic rule application

Dual-Load Cognitive Challenge

This task implements a straightforward dual-load design where difficulty scales along two independent dimensions:

Load Dimension 1: Rule Complexity

  • More Rules = Harder: Additional interference rules increase working memory load
  • Rule Interaction: Multiple rules must be applied in hierarchical order
  • Conditional Processing: Each rule may or may not apply based on mathematical conditions

Load Dimension 2: Output Length

  • Longer Sequences = Harder: More terms require sustained attention and error propagation resistance
  • Cumulative Errors: Mistakes in early terms compound through subsequent calculations
  • Working Memory Persistence: Must maintain rule state across multiple generation steps

Distractor Element: Initial Sequence

The starting sequence serves as a red herring - it appears to establish a pattern but is actually randomly generated. Models must ignore this apparent pattern and focus solely on the explicit rules provided.

Test Case Generation

Algorithm Overview

The generator creates mathematical reasoning scenarios through a systematic process:

  1. Base Rule Selection: Choose fundamental pattern (arithmetic, geometric, square, fibonacci-like)
  2. Interference Rule Selection: Select conditional rules from enabled rule types
  3. Parameter Generation: Generate random parameters for all selected rules
  4. Random Starting Sequence: Create random initial sequence (serves as distractor)
  5. Term Generation: Apply hierarchical rule system to generate target terms
  6. Problem Formatting: Present rules and starting sequence for model continuation

Rule Type System

The system implements 3 categories of interference rules using power-of-2 encoding for flexible combination:

Rule Category Bitmask Description Example Rules
SKIP 1 Rules that replace base pattern under conditions Divisible by N, Contains digit, Prime numbers
MODULO 2 Rules that modify based on position in sequence Every Nth term, Odd positions, Periodic bonuses
CONDITIONS 4 Rules that modify based on previous term properties Even/odd previous, Threshold wrapping, Digit sum

Base Rule System

Four fundamental pattern types establish the core sequence behavior:

Arithmetic Progression

  • Pattern: Add constant step each term
  • Parameters: step (2-8)
  • Example: "Add 3 each time" → 1, 4, 7, 10, 13...

Geometric Progression

  • Pattern: Multiply by constant factor each term
  • Parameters: multiplier (2-4)
  • Example: "Multiply by 2 each time" → 1, 2, 4, 8, 16...

Square Progression

  • Pattern: Square the previous term
  • Parameters: None
  • Example: "Square the previous term" → 2, 4, 16, 256...

Fibonacci-like Progression

  • Pattern: Sum last two terms with offset
  • Parameters: offset (1-3)
  • Example: "Sum last two terms, subtract 1" → 1, 2, 2, 3, 4, 6...

Interference Rule Categories

Skip Rules (Conditional Replacement)

Skip rules replace the base pattern result when specific conditions are met:

Rule Type Condition Action Example
Divisible Skip Result divisible by N Add amount to previous value instead "If divisible by 3, add 2 to previous instead"
Digit Contains Result contains specific digit Add extra amount "If contains digit 7, add 5 extra"
Prime Skip Result is prime number Multiply by factor "If prime, multiply by 2"

Modulo Rules (Position-Based Modification)

Modulo rules modify the base pattern result based on position in sequence:

Rule Type Condition Action Example
Every Nth Add Position divisible by N Add bonus amount "Every 3 terms, add 4 extra"
Odd Position Position is odd Add/subtract modifier "On odd positions, subtract 2"

Conditional Rules (State-Dependent Modification)

Conditional rules modify the base pattern result based on previous term properties:

Rule Type Condition Action Example
Even Condition Previous term was even Add bonus "If previous even, add 3 extra"
Threshold Wrap Result exceeds threshold Wrap to small value "If exceeds 50, wrap to 5"
Digit Sum Sum of digits exceeds limit Subtract amount "If digit sum > 10, subtract 8"

Rule Application Hierarchy

Processing Order

Rules are applied in strict hierarchical order to ensure consistent results:

  1. Base Rule Application: Apply fundamental pattern to generate initial result
  2. Skip Rule Processing: Check if skip conditions are met; if so, replace result entirely
  3. Modulo Rule Processing: Apply position-based modifications to current result
  4. Conditional Rule Processing: Apply state-dependent modifications to current result

Example Rule Interaction

Starting sequence: [2, 5]
Base rule: "Add 3 each time"
Skip rule: "If divisible by 4, add 1 to previous instead"
Modulo rule: "Every 2nd term, add 2 extra"
Conditional rule: "If previous was odd, multiply by 2"

Term 3 generation:
1. Base: 5 + 3 = 8
2. Skip: 8 divisible by 4? Yes → 5 + 1 = 6
3. Modulo: Position 3, every 2nd? No → 6
4. Conditional: Previous (5) was odd? Yes → 6 × 2 = 12
Result: 12

Configuration Parameters

Generation Schema (SequenceGenerationParams)

class SequenceGenerationParams(BaseModel):
    count: int                                  # Number of test cases to generate (> 0)
    seq_length: int                            # Number of terms to generate (≥ 1)
    num_rules: int                             # Number of interference rules (≥ 1)
    rule_enable: int                           # Bitmask of enabled rule types (default: 7)

Standard Complexity Progression: - seq_length: [1, 2, 3, 4, 5] - Generation complexity - num_rules: [1, 2, 3] - Rule interaction complexity - rule_enable: Subset of 3 rule categories - Reasoning complexity

Result Schema (SequenceTestCaseResult)

class SequenceTestCaseResult(BaseModel):
    input: str                                 # Formatted problem text
    target: str                                # Space-separated next terms
    starting_sequence: List[int]               # Initial sequence values
    rules: List[str]                           # Rule descriptions in English
    expected_next_terms: List[int]             # Target sequence terms
    seq_length: int                            # Number of terms generated
    depth: int                                 # Total rules (base + interference)

Example Test Cases

Basic Arithmetic with Skip Rule (seq_length=3, num_rules=1, rules=[SKIP])

Starting sequence: 2 5 8
Rules:
  1. Add 3 each time
  2. If result is divisible by 6, add 2 to previous sequence value instead of following the base pattern
Return the next 3 terms

Rule Application: - Term 4: 8 + 3 = 11 (not divisible by 6) → 11 - Term 5: 11 + 3 = 14 (not divisible by 6) → 14
- Term 6: 14 + 3 = 17 (not divisible by 6) → 17

Expected Answer: "11 14 17"

Geometric with Multiple Interference (seq_length=2, num_rules=2, rules=[MODULO, CONDITIONS])

Starting sequence: 1 2 4
Rules:
  1. Multiply by 2 each time
  2. Every 3 terms, add 5 extra
  3. If previous term was even, add 3 extra
Return the next 2 terms

Rule Application: - Term 4: 4 × 2 = 8 → Position 4 (not every 3rd) → Previous (4) even, add 3 → 8 + 3 = 11 - Term 5: 11 × 2 = 22 → Position 5 (not every 3rd) → Previous (11) odd, no bonus → 22

Expected Answer: "11 22"

Complex Multi-Rule Interaction (seq_length=4, num_rules=3, rules=[SKIP, MODULO, CONDITIONS])

Starting sequence: 3 6
Rules:
  1. Add 4 each time
  2. If result contains digit 1, add 7 extra
  3. Every 2 terms, add 3 extra
  4. If result exceeds 25, wrap around to 2
Return the next 4 terms

Rule Application: - Term 3: 6 + 4 = 10 → Contains '1', add 7 → 17 → Position 3 (not every 2nd) → Under 25 → 17 - Term 4: 17 + 4 = 21 → No '1' → Position 4 (every 2nd), add 3 → 24 → Under 25 → 24 - Term 5: 24 + 4 = 28 → No '1' → Position 5 (not every 2nd) → Exceeds 25, wrap → 2 - Term 6: 2 + 4 = 6 → No '1' → Position 6 (every 2nd), add 3 → 9 → Under 25 → 9

Expected Answer: "17 24 2 9"

Cognitive Skills Tested

Primary Mathematical Capabilities

  • Rule Application: Systematically applying multiple mathematical rules in sequence
  • Conditional Logic: Processing if-then mathematical reasoning
  • Mathematical Operations: Handling arithmetic, geometric, and custom mathematical functions
  • Sequential Processing: Maintaining accuracy across multiple term generations
  • Working Memory: Tracking multiple rules and parameters simultaneously

Executive Function Skills

  • Attention Control: Ignoring distracting initial sequence patterns
  • Rule Prioritization: Understanding hierarchical rule application order
  • Systematic Processing: Following structured procedures without shortcuts
  • Error Resistance: Maintaining accuracy as sequence length increases

Applications

This test manifold evaluates capabilities essential for:

  • Mathematical Reasoning: Applying complex mathematical rule systems
  • Algorithmic Thinking: Following structured procedures with conditional logic
  • Sequential Processing: Generating accurate sequences through multi-step rule application
  • Working Memory Management: Handling increasing cognitive load gracefully
  • Attention Control: Focusing on explicit instructions while ignoring distractors