Skip to content

Shape Test Manifold

Overview

The Shape test manifold evaluates a model's ability to recognize geometric shapes from their SVG path representations, requiring spatial reasoning, geometric pattern recognition, and invariance to transformations. Models must identify shape types from mathematical path descriptions while handling rotations, scaling, translations, and coordinate system variations that preserve geometric identity but alter numerical representations.

Task Description

Models are presented with SVG path elements containing mathematical descriptions of geometric shapes and must identify the underlying shape type. The task requires parsing complex coordinate sequences, understanding geometric relationships, and recognizing shape patterns despite mathematical transformations that preserve geometric properties while changing numerical values.

Key Features: - Geometric Recognition: Identifying shape types from mathematical path descriptions - Transformation Invariance: Recognizing shapes despite rotation, scaling, and translation - Coordinate Parsing: Processing complex SVG path syntax with multiple command types - Spatial Reasoning: Understanding geometric relationships from numerical coordinates - Pattern Recognition: Identifying consistent geometric properties across variations - Mathematical Robustness: Handling floating-point precision and coordinate variations

Test Case Generation

Algorithm Overview

The generator creates challenging shape recognition scenarios through a systematic process:

  1. Shape Selection: Choose from 15 geometric shape types with varying complexity levels
  2. Parameter Generation: Create random dimensional parameters within shape-specific constraints
  3. Base Path Construction: Generate fundamental SVG path commands for the shape
  4. Transformation Application: Apply random rotations, scaling, and translations
  5. Coordinate Calculation: Compute transformed coordinates using trigonometric functions
  6. SVG Path Formatting: Convert mathematical coordinates to standard SVG path syntax
  7. Precision Handling: Format floating-point coordinates with appropriate precision

Shape Type Hierarchy

The system organizes shapes by geometric complexity and recognition difficulty:

Complexity Shape Types Count Recognition Challenge
Level 2 Arc 1 Partial curves, angle interpretation
Level 3 Triangle, Sector 2 Basic polygons, circular sectors
Level 4 Rectangle, Square, Circle, Ellipse 4 Fundamental geometric primitives
Level 5 Parallelogram, Rhombus, Kite, Trapezoid, Moon, Pentagon, House, Arrow 8 Complex polygons, composite shapes

Shape Definitions and Parameters

Level 2 Shapes

Arc (Complexity: 2) - Parameters: radius (15-50), angle (30-180°) - Recognition Challenge: Partial circular curves without closure - Path Structure: Move + Arc command - Example: M 25.00 0.00 A 25.00 25.00 0.00 0 1 17.68 17.68

Level 3 Shapes

Triangle (Complexity: 3) - Parameters: base (20-60), height (20-60) - Recognition Challenge: Three-sided polygon with variable proportions - Path Structure: Move + 2 Lines + Close - Example: M 0.00 0.00 L 40.00 0.00 L 20.00 35.00 Z

Sector (Complexity: 3) - Parameters: radius (15-50), angle (30-270°) - Recognition Challenge: Pie-slice shapes with varying angles - Path Structure: Move + Line + Arc + Close - Example: M 0.00 0.00 L 30.00 0.00 A 30.00 30.00 0.00 0 1 21.21 21.21 Z

Level 4 Shapes

Rectangle (Complexity: 4) - Parameters: width (20-80), height (20-80) - Recognition Challenge: Four-sided orthogonal shapes - Path Structure: Move + 3 Lines + Close - Example: M 0.00 0.00 L 60.00 0.00 L 60.00 40.00 L 0.00 40.00 Z

Square (Complexity: 4) - Parameters: size (20-60) - Recognition Challenge: Equal-sided rectangles - Path Structure: Move + 3 Lines + Close - Example: M 0.00 0.00 L 40.00 0.00 L 40.00 40.00 L 0.00 40.00 Z

Circle (Complexity: 4) - Parameters: radius (15-50) - Recognition Challenge: Perfect circular shapes using two arc commands - Path Structure: Move + 2 Arcs + Close - Example: M -25.00 0.00 A 25.00 25.00 0.00 0 1 25.00 0.00 A 25.00 25.00 0.00 0 1 -25.00 0.00 Z

Ellipse (Complexity: 4) - Parameters: rx (15-50), ry (15-50) - Recognition Challenge: Oval shapes with different radii - Path Structure: Move + 2 Arcs + Close - Example: M -30.00 0.00 A 30.00 20.00 0.00 0 1 30.00 0.00 A 30.00 20.00 0.00 0 1 -30.00 0.00 Z

Level 5 Shapes

Parallelogram (Complexity: 5) - Parameters: width (30-70), height (20-50), skew (10-30) - Recognition Challenge: Skewed rectangles with parallel sides - Path Structure: Move + 3 Lines + Close - Example: M 0.00 0.00 L 50.00 0.00 L 65.00 30.00 L 15.00 30.00 Z

Rhombus (Complexity: 5) - Parameters: side_length (25-55), angle (45-135°) - Recognition Challenge: Equal-sided parallelograms - Path Structure: Move + 3 Lines + Close - Trigonometric Calculations: Uses cosine/sine for vertex positioning

Kite (Complexity: 5) - Parameters: top_length (20-50), bottom_length (20-50), width (15-45) - Recognition Challenge: Quadrilaterals with two pairs of adjacent equal sides - Path Structure: Move + 3 Lines + Close - Example: M 0.00 0.00 L 25.00 35.00 L 0.00 75.00 L -25.00 35.00 Z

Trapezoid (Complexity: 5) - Parameters: bottom_width (40-80), top_width (20-60), height (20-50) - Recognition Challenge: Quadrilaterals with one pair of parallel sides - Path Structure: Move + 3 Lines + Close

Moon (Complexity: 5) - Parameters: outer_radius (25-50), inner_radius (15-35), offset (5-15) - Recognition Challenge: Crescent shapes using opposing arcs - Path Structure: Move + 2 Arcs + Close - Geometric Complexity: Requires understanding of arc intersection

Pentagon (Complexity: 5) - Parameters: radius (20-50) - Recognition Challenge: Regular five-sided polygon - Path Structure: Move + 4 Lines + Close - Trigonometric Calculations: Uses 2π/5 radians for vertex positioning

House (Complexity: 5) - Parameters: width (40-70), wall_height (30-50), roof_height (20-40) - Recognition Challenge: Composite shape (rectangle + triangle) - Path Structure: Move + 4 Lines + Close - Example: M 0.00 0.00 L 55.00 0.00 L 55.00 40.00 L 27.50 70.00 L 0.00 40.00 Z

Arrow (Complexity: 5) - Parameters: shaft_length (30-60), shaft_width (8-15), head_width (20-35), head_length (15-25) - Recognition Challenge: Composite directional shape - Path Structure: Move + 6 Lines + Close - Geometric Complexity: Requires understanding of shaft and arrowhead proportions

Transformation System

Coordinate Transformation Mathematics

The generator applies mathematical transformations that preserve geometric identity while changing numerical representations:

Transformation Matrix Application: 

x' = x * scale * cos(θ) - y * scale * sin(θ) + offset_x
y' = x * scale * sin(θ) + y * scale * cos(θ) + offset_y

Where: - θ = rotation angle in radians - scale = uniform scaling factor - offset_x, offset_y = translation offsets

Transformation Types

Rotation (rotate_min to rotate_max degrees) - Geometric Invariance: Shape identity preserved under rotation - Numerical Challenge: All coordinates change despite identical shape - Recognition Test: Models must recognize rotated patterns - Example: Square rotated 45° appears as diamond in coordinates

Scaling (scale_min to scale_max factor) - Geometric Invariance: Proportional relationships preserved - Numerical Challenge: All dimensions change proportionally - Recognition Test: Models must recognize size-independent patterns - Example: Circle with radius 25 vs radius 50 (scale factor 2.0)

Translation (offset_min to offset_max units) - Geometric Invariance: Shape and size preserved - Numerical Challenge: All coordinates shifted by offset - Recognition Test: Models must ignore absolute positioning - Example: Triangle at origin vs triangle at (100, 50)

Arc Command Transformations

Special handling for SVG arc commands during transformation: - Radii Scaling: rx' = rx * scale, ry' = ry * scale - Rotation Composition: x_axis_rotation' = (x_axis_rotation + rotation) % 360 - Endpoint Transformation: Arc endpoints transformed like regular coordinates - Flag Preservation: large_arc_flag and sweep_flag remain unchanged

Configuration Parameters

Generation Schema (ShapeGenerationParams)

class ShapeGenerationParams(BaseModel):
    count: int                                   # Number of test cases to generate (> 0)
    shape_type: ShapeType                       # Specific shape type to generate (enum)
    rotate_min: float                           # Minimum rotation in degrees (default: 0)
    rotate_max: float                           # Maximum rotation in degrees (default: 0)
    scale_min: float                            # Minimum scaling factor (default: 1.0)
    scale_max: float                            # Maximum scaling factor (default: 1.0)
    offset_min: float                           # Minimum position offset (default: 0)
    offset_max: float                           # Maximum position offset (default: 0)

Result Schema (ShapeTestCaseResult)

class ShapeTestCaseResult(BaseModel):
    input: str                                  # The SVG path element
    target: str                                 # The shape name
    rotation: float                             # Applied rotation in degrees
    scale: float                                # Applied scaling factor
    offset_x: float                             # Applied X offset
    offset_y: float                             # Applied Y offset

Example Test Cases

Basic Shape Recognition (shape_type=CIRCLE, no transformations)

Input: <path d="M -25.00 0.00 A 25.00 25.00 0.00 0 1 25.00 0.00 A 25.00 25.00 0.00 0 1 -25.00 0.00 Z"/>
Target: circle

Path Analysis: - M -25.00 0.00: Move to leftmost point of circle - A 25.00 25.00 0.00 0 1 25.00 0.00: Arc from left to right (semicircle) - A 25.00 25.00 0.00 0 1 -25.00 0.00: Arc from right to left (semicircle) - Z: Close path - Recognition Pattern: Two 180° arcs with equal radii = circle

Rotation Challenge (shape_type=SQUARE, rotation=45°)

Input: <path d="M -14.14 14.14 L 14.14 42.42 L 42.42 14.14 L 14.14 -14.14 Z"/>
Target: square

Transformation Analysis: - Original Square: (0,0) → (40,0) → (40,40) → (0,40) - 45° Rotation: Coordinates rotated around origin - Recognition Challenge: Diamond orientation obscures square pattern - Geometric Invariance: Equal sides and right angles preserved

Scaling Challenge (shape_type=TRIANGLE, scale=2.5)

Input: <path d="M 0.00 0.00 L 100.00 0.00 L 50.00 87.50 Z"/>
Target: triangle

Scaling Analysis: - Original Triangle: base=40, height=35 - Scaled Triangle: base=100, height=87.5 (2.5x scaling) - Recognition Challenge: Larger dimensions, same proportions - Geometric Invariance: Angle relationships preserved

Translation Challenge (shape_type=PENTAGON, offset=(50, 30))

Input: <path d="M 80.00 30.00 L 69.27 53.09 L 45.11 53.09 L 34.38 30.00 L 50.00 12.36 Z"/>
Target: pentagon

Translation Analysis: - Original Pentagon: Centered at origin - Translated Pentagon: All coordinates shifted by (50, 30) - Recognition Challenge: Absolute positioning changed - Geometric Invariance: Shape and proportions identical

Complex Transformation (shape_type=ARROW, rotation=30°, scale=1.5, offset=(25, 15))

Input: <path d="M 25.00 8.75 L 90.45 28.99 L 90.45 21.49 L 107.99 36.25 L 90.45 51.01 L 90.45 43.51 L 25.00 21.25 Z"/>
Target: arrow

Multi-Transformation Analysis: - Base Arrow: shaft_length=45, shaft_width=10, head_width=25, head_length=18 - Applied Transformations: 30° rotation + 1.5x scaling + (25,15) translation - Recognition Challenge: Multiple simultaneous transformations - Coordinate Complexity: All vertices affected by compound transformation

Arc Recognition Challenge (shape_type=ARC, rotation=60°, scale=0.8)

Input: <path d="M 10.00 17.32 A 20.00 20.00 60.00 0 1 -6.93 28.00"/>
Target: arc

Arc Transformation Analysis: - Original Arc: radius=25, angle=90° - Transformed Arc: radius=20 (0.8x scale), rotation=60° - Arc Command Changes: Radii scaled, x_axis_rotation updated - Recognition Challenge: Partial curve without clear geometric closure

SVG Path Command Reference

Command Types Used

Command Description Parameters Usage
M Move To x, y Starting point of path
L Line To x, y Straight line to point
A Arc To rx, ry, x-axis-rotation, large-arc-flag, sweep-flag, x, y Elliptical arc
Z Close Path (none) Close current subpath

Arc Command Complexity

Arc commands represent the most complex recognition challenge: - rx, ry: Ellipse radii (scaled during transformation) - x-axis-rotation: Ellipse rotation (updated during transformation) - large-arc-flag: 0 for ≤180°, 1 for >180° (preserved) - sweep-flag: 0 for counterclockwise, 1 for clockwise (preserved) - x, y: Arc endpoint (transformed like regular coordinates)

Cognitive Skills Tested

  • Geometric Pattern Recognition: Identifying shape types from coordinate sequences
  • Spatial Reasoning: Understanding geometric relationships in coordinate space
  • Transformation Invariance: Recognizing shapes despite mathematical transformations
  • Mathematical Parsing: Processing complex SVG path syntax
  • Coordinate System Understanding: Interpreting 2D coordinate relationships
  • Trigonometric Reasoning: Understanding rotational transformations
  • Scale Invariance: Recognizing proportional relationships across size changes
  • Path Interpretation: Following sequential drawing commands to visualize shapes
  • Precision Handling: Managing floating-point coordinate variations
  • Composite Shape Analysis: Recognizing complex shapes built from basic elements

Applications

This test manifold evaluates capabilities essential for:

  • Computer Graphics: Understanding vector graphics and path-based shape representations
  • CAD Systems: Recognizing geometric shapes in technical drawings
  • Image Processing: Shape detection and classification in visual data
  • Mathematical Visualization: Interpreting geometric data from numerical descriptions
  • Spatial Analysis: Understanding coordinate-based geometric information
  • Vector Graphics Processing: Working with SVG, PostScript, and similar formats
  • Geometric Modeling: Recognizing and manipulating mathematical shape descriptions
  • Technical Drawing Interpretation: Understanding engineering and architectural drawings