Skip to content
PoseFlow docs

Architecture

PoseFlow runs the same five-stage pipeline on every surface that consumes the SDK. This page explains those stages and how they hand off.

The pipeline

                ┌───────────────────────────────┐
   1. CAMERA   │  PoseCameraView (native)       │  CameraTrackingView (web)
                │  CameraImage stream            │  HTMLVideoElement + Worker
                └────────────┬───────────────────┘
                             │ raw frames
                             v
                ┌───────────────────────────────┐
   2. POSE     │  Native pose engine       │  Web Worker (the pose engine WASM)
                │  pure-C FFI                    │  same model, different host
                └────────────┬───────────────────┘
                             │ Pose (33 landmarks + worldZ + vis)
                             │, display-space, [0, 1] normalised
                             v
                ┌───────────────────────────────┐
   3. TRACK    │  MovementTracker               │
                │   - PhaseStateMachine          │  phase transitions
                │   - rep counter + scoring gate │
                │   - form-rule evaluator        │
                │   - CameraAngleDetector        │  16-bucket orientation
                │   - frame validator            │  visibility / distance gates
                └────────────┬───────────────────┘
                             │ TrackingResult (per frame)
                             │ + RepCompletedEvent stream (on rep)
                             │ + FormFeedbackEvent stream (on cue)
                             v
                ┌───────────────────────────────┐
   4. STATE    │  Your app state (BLoC, etc.)
                │   - rep count                  │  → UI counter
                │   - form score                 │  → UI score badge
                │   - feedback events            │  → toast deck / haptics
                │   - detected phase             │  → highlighted phase tile
                │   - detected bucket            │  → angle indicator
                └────────────┬───────────────────┘
                             │
                             v
                ┌───────────────────────────────┐
   5. RENDER   │  Your widgets                  │
                └───────────────────────────────┘

The contract between stages

Each stage talks to the next via a small, well-typed surface:

StageOutputType
CameraRaw frameCameraImage (native) / ImageBitmap (web)
PoseDetected posePose
Track (per frame)Tracking resultTrackingResult
Track (events)Rep + feedback eventsRepCompletedEvent, FormFeedbackEvent

Your app never reaches into the tracker; it consumes the two streams and the per-frame result and renders.

The canonical pose coordinate space

PoseFlow exposes one pose space across the whole pipeline: display-space, [0, 1] normalised against the post-rotation buffer dimensions, selfie-mirrored on the front camera.

  • The skeleton overlay paints in display space.
  • MovementTracker evaluates tracking points (angle, distance, ratio, position, velocity, stability) in display space.
  • PoseFlow Studio authors .pose thresholds against the same display-space values its preview tracker sees.

Authoring (Studio) and playback (your app) agree on coordinate scale by construction, distance / position / ratio thresholds resolve against the same numerical values on both sides.

The canonical entry point

Load movements through one call:

final tracker = MovementTracker.withPreset(MovementTrackerPreset.standard);
tracker.load(
  movement,                  // a Movement decoded from .pose
  configOverride: cfg,       // optional published MovementConfig overlay
);

If you use TrackedMovementView, the widget handles this for you, pass the movement and any optional config, the widget loads them.

Rep-counting modes

MovementTracker runs in one of two modes depending on how the Movement was authored:

  • Rule-based phases (phaseConfigs populated, positions empty) , the default, and the only mode PoseFlow Studio emits. The PhaseStateMachine advances when each phase’s authored conditions (membership gates) evaluate to true against the current per-frame tracking values; reps complete when the phase machine cycles through the authored sequence.
  • Vector matching (Movement.positions.isNotEmpty), for movements built from a recorded marker pass. A small PCA index classifies the current pose against named reference positions. The PhaseStateMachine ticks when the user traverses the authored sequence of positions.

PoseFlow Studio always emits the rule-based shape, declarative phase conditions, no recorded reference frames. The vector-matching mode is available for movements built outside Studio against a recorded marker pass.

Camera-angle bucketing

The 16-bucket CameraAngleDetector is a sidecar that classifies the user’s body orientation every frame into one of {front, behind, ±45°, ±90°} × {hip, floor, overhead}. Its output gates the form service: when an movement has multiple AngleBands (one set of ranges per camera bucket), the tracker uses the detected bucket to pick the right reference. Read more in camera buckets.

Where the SDK ends and the Studio starts

The Studio is purely an authoring tool, it never participates in the runtime tracking path. The Studio uses the same MovementTracker (via its ShowcaseTracker facade) to drive its live preview, so “what you author is what the user gets” is enforced by construction: any movement that ticks reps in the Studio will tick reps in your app.

When the Studio saves, it serialises the in-memory Movement to JSON via Movement.toJson(). Loading is the inverse , Movement.fromJson(). No special “studio file” format exists; the .pose file is the canonical artifact and any SDK consumer can read it without involving the Studio at all.