Part IV: Runtime System

Version: 1.0 Draft Last Updated: 2025-12-20

1. Introduction

The runtime system executes the graph constructed during wiring. It manages:

Object lifecycle (initialization, start, stop, disposal)
Time progression and scheduling
Evaluation ordering and execution
Event propagation and notification

        graph TB
    subgraph "Runtime Components"
        G[Graph]
        SCH[Scheduler]
        CLK[Clock]
        ENG[Engine]
        N1[Node 1]
        N2[Node 2]
        N3[Node 3]
    end

    ENG --> SCH
    ENG --> CLK
    ENG --> G
    G --> N1
    G --> N2
    G --> N3
    SCH --> |"schedule"| N1
    SCH --> |"schedule"| N2
    CLK --> |"current time"| ENG

2. Lifecycle

All runtime components follow a consistent lifecycle:

2.1 State Machine

        stateDiagram-v2
    [*] --> Constructed
    Constructed --> Initialized: initialise()
    Initialized --> Started: start()
    Started --> Evaluating: eval()
    Evaluating --> Started: tick complete
    Started --> Stopped: stop()
    Stopped --> Started: start()
    Stopped --> Disposed: dispose()
    Disposed --> [*]

2.2 Lifecycle Methods

Method	Description	Called When
`initialise()`	One-time setup, allocate resources	Before first start
`start()`	Begin active operation	Each run/resume
`eval()`	Perform computation	Each scheduled tick
`stop()`	Pause operation	End run or pause
`dispose()`	Release all resources	Final cleanup

2.3 Lifecycle Guarantees

Guarantee	Description
Single Init	`initialise()` called exactly once
Matched Start/Stop	Every `start()` has matching `stop()`
Order	`initialise` → `start` → (eval)* → `stop` → `dispose`
Topological	Nodes initialized/started in dependency order
Reverse Dispose	Nodes disposed in reverse dependency order

3. Graph

The Graph is the top-level runtime container:

3.1 Graph Structure

        classDiagram
    class Graph {
        +parent_node: Node | None
        +graph_id: str
        +nodes: Sequence[Node]
        +evaluation_clock: EvaluationClock
        +engine_evaluation_clock: EngineEvaluationClock
        +evaluation_engine: EvaluationEngine
        +evaluation_time: datetime
        +last_evaluation_time: datetime
        +is_started: bool
        +initialise()
        +start()
        +stop()
        +dispose()
        +schedule_node(node_ndx, when, force_set)
        +evaluate_graph()
    }

    class Node {
        +graph: Graph
        +inputs: list[Input]
        +output: Output
        +eval()
    }

    class EvaluationClock {
        +evaluation_time: datetime
        +now: datetime
        +cycle_time: timedelta
        +next_cycle_evaluation_time: datetime
    }

    Graph "1" --> "*" Node
    Graph "1" --> "1" EvaluationClock

3.2 Graph Properties

Property	Type	Description
`parent_node`	`Node	None`
`graph_id`	`str`	Unique graph identifier
`nodes`	`Sequence[Node]`	All nodes in topological order
`evaluation_time`	`datetime`	Current tick’s time
`last_evaluation_time`	`datetime`	Time of last evaluation
`evaluation_clock`	`EvaluationClock`	User-facing clock
`engine_evaluation_clock`	`EngineEvaluationClock`	Engine clock (internal)
`evaluation_engine`	`EvaluationEngine`	Evaluation engine
`evaluation_engine_api`	`EvaluationEngineApi`	Engine API interface
`traits`	`GraphTraits`	Graph configuration traits
`receiver`	`_SenderReceiverState`	Push source event receiver
`schedule`	`list[datetime]`	Scheduled evaluation times
`is_started`	`bool`	True if graph is running

4. Scheduler

The Scheduler manages when nodes are evaluated:

4.1 Scheduling Model

        graph TB
    subgraph "Scheduler"
        PQ[Priority Queue<br/>time → nodes]
        NOW[Current Time]
        READY[Ready Queue]
    end

    N1[Node 1] --> |"schedule_node(t1)"| PQ
    N2[Node 2] --> |"schedule_node(t2)"| PQ
    N3[Node 3] --> |"notify()"| READY

    PQ --> |"pop at time"| READY
    READY --> |"evaluate"| ENG[Engine]

4.2 Scheduling Operations

Operation	Description
`schedule(node, time)`	Add node to be evaluated at `time`
`pop_scheduled_nodes(time)`	Get all nodes scheduled for `time`
`next_scheduled_time()`	Return next time with pending nodes
`has_pending()`	True if any nodes are scheduled

4.3 Priority Ordering

Within a single tick, nodes are ordered by:

Topological rank (sources before dependents)
Node ID (for deterministic ordering within same rank)

        graph LR
    subgraph "Tick at T=100"
        direction LR
        R1[Rank 0<br/>Sources]
        R2[Rank 1<br/>Compute]
        R3[Rank 2<br/>Compute]
        R4[Rank 3<br/>Sinks]
    end

    R1 --> R2 --> R3 --> R4

5. Clock

The Clock provides time management:

5.1 Clock Types

        graph TD
    CLK[Clock]

    CLK --> SIM[Simulation Clock]
    CLK --> RT[Real-Time Clock]

    SIM --> |"Advances as fast as possible"| FAST[Fast-forward]
    RT --> |"Tracks wall clock"| WALL[Wall Clock]

5.2 Clock Properties

Property	Description
`now`	Current time
`evaluation_time`	Time being evaluated
`push_time`	Source of async events
`cycle_id`	Current evaluation cycle ID

5.3 Time Semantics

Mode	Time Behavior
Simulation	Jumps directly to next scheduled event
Real-Time	Sleeps until next scheduled event

6. Evaluation Loop

6.1 Main Loop

        sequenceDiagram
    participant E as Engine
    participant S as Scheduler
    participant G as Graph
    participant N as Nodes

    E->>S: Initialize
    loop Until end_time or stop
        E->>S: next_scheduled_time()
        S-->>E: next_time
        alt Simulation Mode
            E->>G: Set evaluation_time = next_time
        else Real-Time Mode
            E->>E: Sleep until next_time
            E->>G: Set evaluation_time = now()
        end
        E->>S: pop_scheduled_nodes()
        S-->>E: nodes_to_evaluate
        loop For each node in topological order
            E->>N: eval()
            N->>N: Read inputs
            N->>N: Compute
            N->>N: Write output
            N-->>E: May schedule more nodes
        end
        E->>E: Increment cycle
    end

6.2 Evaluation Phases Per Tick

Phase	Description
1. Collect	Get all nodes scheduled for current time
2. Sort	Order by topological rank
3. Evaluate	Call `eval()` on each node
4. Propagate	Handle notifications and scheduling
5. Cleanup	Reset per-tick state

6.3 Node Evaluation

        sequenceDiagram
    participant E as Engine
    participant N as Node
    participant I as Inputs
    participant O as Output

    E->>N: eval()
    N->>I: Check modified inputs
    alt Any active input modified
        N->>I: Read values
        N->>N: Execute function
        alt Has output
            N->>O: Write result
            O->>O: Mark modified
            O->>E: Notify dependents
        end
    end

7. Notification System

7.1 Notification Flow

        graph TB
    O[Output Modified]
    O --> N1[Notify Bound Inputs]
    N1 --> N2[Check Active Status]
    N2 --> |"Active"| S[Schedule Node]
    N2 --> |"Passive"| SKIP[Skip]
    S --> Q[Add to Scheduler Queue]

7.2 Notification Idempotency

Notifications are deduplicated per evaluation time:

def notify(self, evaluation_time):
    if self._notify_time != evaluation_time:
        self._notify_time = evaluation_time
        self._propagate_to_parent()
        self._schedule_node()

7.3 Parent Chain Notification

For nested time-series (bundles, collections):

        graph TB
    C[Child TS Modified]
    C --> P1[Parent TSB Field]
    P1 --> P2[Parent TSB]
    P2 --> N[Owning Node Input]
    N --> S[Scheduler]

8. Input/Output Binding

8.1 Binding Model

        graph LR
    subgraph "Node A"
        OA[Output]
    end

    subgraph "Node B"
        IB[Input]
    end

    subgraph "Node C"
        IC[Input]
    end

    OA --> |"bind"| IB
    OA --> |"bind"| IC

8.2 Binding Operations

Operation	Description
`bind_output(output)`	Connect input to output
`unbind_output()`	Disconnect from output
`bound`	True if connected to an output

8.3 Binding Effects

When binding occurs:

Input marks itself as bound
Input subscribes to output notifications
If output is valid, input copies current value
If input is active, node is scheduled

9. Active/Passive Subscriptions

9.1 Subscription Model

State	Behavior
Active	Input triggers node evaluation on modification
Passive	Input receives updates but doesn’t trigger evaluation

9.2 State Transitions

        stateDiagram-v2
    [*] --> Active: default
    Active --> Passive: make_passive()
    Passive --> Active: make_active()

9.3 Subscription Use Cases

@compute_node
def sample(trigger: TS[bool], value: TS[int]) -> TS[int]:
    """Only emit when trigger fires, using current value."""
    # 'value' could be passive - only read when trigger fires
    return value.value

10. Modification Tracking

10.1 Tracking Invariant

modified = (last_modified_time == evaluation_time)

A time-series is modified if it was changed in the current tick.

10.2 Tracking Implementation

Property	Description
`last_modified_time`	Time of most recent modification
`modified`	Computed: `last_modified_time == now`

10.3 Modification Propagation

        sequenceDiagram
    participant V as Value
    participant O as Output
    participant I as Inputs
    participant N as Nodes

    V->>O: set_value()
    O->>O: Update last_modified_time
    O->>O: Mark modified
    O->>I: Notify bound inputs
    I->>N: Schedule if active

11. Error Handling

11.1 Error Types

Error Type	Description
`GraphError`	General graph errors
`NodeError`	Node-specific errors
`SchedulerError`	Scheduling failures
`EvaluationError`	Runtime evaluation failures

11.2 Error Propagation

        graph TB
    N[Node Error]
    N --> EO[Error Output]
    EO --> EH[Error Handler Node]
    EH --> |"recover"| CONT[Continue]
    EH --> |"fail"| STOP[Stop Graph]

11.3 Error Outputs

Nodes can have optional error outputs:

@compute_node
def risky_operation(x: TS[int]) -> TSB[ValueWithError]:
    try:
        result = complex_calculation(x.value)
        return {"value": result, "error": None}
    except Exception as e:
        return {"value": None, "error": str(e)}

12. Execution Modes

12.1 Simulation Mode

        graph LR
    T1[t=0] --> |"instant"| T2[t=100]
    T2 --> |"instant"| T3[t=500]
    T3 --> |"instant"| T4[t=1000]

Characteristics:

Time jumps to next scheduled event
No push sources allowed
Deterministic execution
Reproducible results

12.2 Real-Time Mode

        graph LR
    T1[t=0] --> |"sleep 100ms"| T2[t=100]
    T2 --> |"sleep 400ms"| T3[t=500]
    T3 --> |"sleep 500ms"| T4[t=1000]

Characteristics:

Time tracks wall clock
Push sources allowed
Non-deterministic (external events)
May process late events

12.3 Mode Comparison

Aspect	Simulation	Real-Time
Time progression	As fast as possible	Wall clock
Push sources	Prohibited	Allowed
Determinism	Guaranteed	Not guaranteed
Reproducibility	Full	Limited
Use case	Backtesting	Live operation

13. Injectable Types

13.1 Available Injectables

Injectable	Description
`STATE[T]`	Mutable state container
`SCHEDULER`	Node scheduling access
`CLOCK`	Time information
`REPLAY_STATE`	State with replay support
`OUTPUT`	Direct output access
`CONTEXT`	Runtime context

13.2 Usage

@compute_node
def stateful_counter(
    trigger: TS[bool],
    state: STATE[int] = 0
) -> TS[int]:
    if trigger.value:
        state.value += 1
    return state.value

13.3 Injection Lifecycle

Injectable	When Provided
`STATE`	Created at initialization
`SCHEDULER`	Available from start
`CLOCK`	Available always

14. Engine Configuration

14.1 Configuration Options

Option	Description
`start_time`	Evaluation start time
`end_time`	Evaluation end time (optional)
`run_mode`	SIMULATION or REAL_TIME
`trace`	Enable execution tracing

14.2 run_graph Function

def run_graph(
    graph: GraphBuilder,
    start_time: datetime = MIN_DT,
    end_time: datetime | None = None,
    run_mode: EvaluationMode = SIMULATION,
    **kwargs
) -> dict[str, Any] | None:
    """Execute a graph."""

15. Thread Safety

15.1 Threading Model

Component	Thread Safety
Scheduler	Single-threaded access
Evaluation	Single evaluation thread
Push sources	Multi-threaded injection
Internal state	Protected by evaluation model

15.2 Push Source Threading

        graph TB
    subgraph "External Threads"
        T1[Thread 1]
        T2[Thread 2]
    end

    subgraph "Engine"
        Q[Event Queue]
        EV[Evaluation Thread]
    end

    T1 --> |"push"| Q
    T2 --> |"push"| Q
    Q --> |"drain"| EV

16. Implementation Notes

16.1 Push Source Node Processing Phase

Push source nodes have special handling in the evaluation loop that occurs BEFORE normal compute nodes:

        sequenceDiagram
    participant EQ as Event Queue
    participant EN as Engine
    participant PN as Push Nodes
    participant CN as Compute Nodes

    Note over EQ,CN: Evaluation Tick Start
    EQ->>EN: Dequeue push messages
    EN->>PN: Apply messages to push nodes
    Note over PN: push_source_nodes_end boundary
    EN->>CN: Evaluate compute nodes
    Note over EQ,CN: Evaluation Tick End

If message application fails, messages are re-queued with backpressure signaling.

16.2 Field-Level Modification Tracking (TSB)

Time-series bundles support field-level modification tracking:

# When a bundle field is modified:
field_output.set(new_value)

# This triggers parent chain notification:
self._parent_or_node.mark_child_modified(self, modified_time)

This enables:

Downstream node scheduling when individual TSB fields change (not just the bundle)
Efficient delta tracking for nested structures
Parent chain notification for TSD and nested bundles

16.3 Node Evaluation Guard Conditions

Nodes evaluate only when ALL of the following are satisfied:

Condition	Description
Required inputs valid	All non-optional inputs have valid values
Collection completeness	For `all_valid=True`, all collection elements valid
Trigger present	Either input modified OR scheduler triggered
Scheduler verification	For scheduler-using nodes, something actually triggered scheduling

16.4 Node-Level Scheduling (NodeSchedulerImpl)

Each node can access its own scheduler via _scheduler: SCHEDULER injectable:

@compute_node
def my_node(ts: TS[int], _scheduler: SCHEDULER = None) -> TS[int]:
    if ts.modified:
        _scheduler.schedule(MIN_TD * 5, tag="delayed")  # Schedule future tick
        return ts.value
    if _scheduler.is_scheduled_now:
        return -1  # Handle scheduled tick

Features:

Sorted event list with optional tags
Alarm support with callbacks
is_scheduled_now property for checking current tick
Wall clock scheduling for real-time mode

17. Reference Locations

Component	Python Location	C++ Location
Graph	`hgraph/_impl/_runtime/_graph.py`	`cpp/src/cpp/runtime/`
Node	`hgraph/_impl/_runtime/_node.py`	`cpp/src/cpp/runtime/`
Scheduler	`hgraph/_impl/_runtime/_scheduler.py`	`cpp/src/cpp/runtime/`
Evaluation Engine	`hgraph/_impl/_runtime/_evaluation_engine.py`	`cpp/src/cpp/runtime/`
Clock	`hgraph/_impl/_runtime/_clock.py`	`cpp/src/cpp/runtime/`

17. Next Steps

Continue to:

05_TIME_SERIES_TYPES.md - Time-series type details
06_NODE_TYPES.md - Node specifications
07_OPERATORS.md - Built-in operators