KEK Logo
KEKDocs

Data Source of Truth

KEK relies on a centralized Data Source of Truth to ensure consistency, auditability, and coordination across the entire system.

This data layer serves as KEK's long-term memory — recording strategy definitions, validation artifacts, performance signals, and lifecycle state in one authoritative record. It enables monitoring, refinement, and reproducibility without granting any execution authority.

What this page covers

  • What the Data Source of Truth is
  • What types of data are stored
  • How data flows through the system
  • Why this layer is critical for safety and transparency

What is the Data Source of Truth?

The Data Source of Truth is the authoritative system of record for all strategy-related state in KEK.

It provides:

  • A single reference point for system coordination and decision context
  • Historical records for evaluation, monitoring, and refinement
  • Separation between intelligence, validation, and execution responsibilities

This aligns with the "single source of truth" concept: aggregating critical state into a unified reference point so all components operate consistently.

Important: This layer does not execute trades and does not control assets.

Technology: TimescaleDB

KEK's Data Source of Truth is built on TimescaleDB, a PostgreSQL extension designed for high-performance time-series analytics.

Why TimescaleDB

  • Full SQL support — Complex joins, analytics, and relational queries alongside time-series data
  • PostgreSQL reliability — Battle-tested database with extensive ecosystem
  • Time-series optimization — Native functions like time_bucket(), first(), last() for financial analysis

Key features

Hypertables

TimescaleDB automatically partitions data by time intervals. Queries with time ranges only scan relevant chunks, enabling fast retrieval even on multi-terabyte datasets.

Compression

Historical data is compressed with 90%+ storage reduction while maintaining query performance. Recent data stays in row format for fast inserts; older data transitions to columnar compression.

Continuous aggregates

Materialized views that refresh automatically as new data arrives. KEK uses these for:

  • Real-time performance dashboards
  • Pre-computed risk metrics (Sharpe, drawdown, win rate)
  • Monitoring signal aggregation

This architecture ensures that strategy evaluation, optimization workflows, and continuous monitoring operate on a consistent, performant data foundation.

What data is stored?

KEK stores structured records that are directly tied to explicit lifecycle stages.

Strategy definitions

  • Strategy specifications (structured, machine-readable)
  • Strategy versions and variant lineage
  • Parameter sets, constraints, and objectives

Validation artifacts

  • Backtesting and optimization results
  • Walk-forward / robustness summaries
  • Stress-test outputs and failure mode notes

Paper trading telemetry

  • Live-simulated performance metrics
  • Behavior diagnostics (timing, exposure, churn)
  • Execution realism assumptions and deltas

Monitoring & drift signals

  • Ongoing performance summaries
  • Drift detection signals (relative + absolute)
  • Regime alignment tracking
  • Alerts and degradation triggers

Refinement actions and adjustments

  • Variant promotion / demotion decisions
  • Re-validation requirements and outcomes

All entries are associated with a lifecycle state, so the system always knows what has been validated, what is being observed, and what is eligible for optional execution.

Why centralization matters

A single authoritative record prevents:

  • Conflicting system state across products and services
  • Silent strategy changes that cannot be explained later
  • Untraceable refinement decisions
  • Inconsistent validation artifacts between environments

This design prioritizes traceability over speed, which is essential in any system that touches capital decisions.

How data flows through the system

KEK data flows through a structured sequence:

  1. Strategy generation writes strategy specs + variants
  2. Validation layers write results (backtests, optimization artifacts)
  3. Paper trading writes live-simulated performance telemetry
  4. Monitoring reads metrics continuously and evaluates drift
  5. Meta-learning reads historical baselines and triggers refinement cycles
  6. Orchestration writes refinement decisions and new versions back into the source of truth

No component operates on hidden or private state. This makes system behavior reproducible and reviewable.

Event lineage and auditability

The Data Source of Truth is designed to preserve lineage: what changed, when it changed, and why.

A common approach to maximizing auditability is using append-only change records (event sourcing patterns), where actions are recorded as immutable events that can reconstruct state and provide an audit trail.

KEK uses this auditability principle to ensure that strategy evolution remains:

  • explainable
  • reversible (rollback-friendly)
  • traceable across time

Role in monitoring and meta-learning

Meta-learning systems depend on consistent historical context to detect drift and justify refinement.

The Data Source of Truth enables:

  • Comparison between strategy variants under the same conditions
  • Drift evaluation against prior baselines
  • Evaluation across regimes and time windows
  • Safe, structured refinement loops grounded in measured outcomes

This is the difference between "continuous improvement" and silent, uncontrolled change.

Relationship to execution

The Data Source of Truth does not:

  • Trigger trades
  • Modify positions
  • Hold funds
  • Bypass user authorization

Execution systems may reference validated strategy state, but execution remains fully separate and user-controlled.

This preserves strict boundaries: data coordination is not execution authority.

Transparency guarantees

This layer supports:

  • Strategy version tracking and provenance
  • Reproducible validation artifacts
  • Historical performance review
  • A clear lineage from generation → validation → monitoring → optional execution

This makes KEK behavior explainable — even retroactively.

Why this matters

This architecture exists to:

  • Reduce operational risk and ambiguity
  • Prevent silent changes
  • Enable disciplined strategy development at scale
  • Support long-term trust through traceability

KEK treats memory as infrastructure — not optimization.

Where this fits in the system

The Data Source of Truth sits beneath validation and monitoring layers, supporting:

Backtesting → Paper Trading → Monitoring & Meta-Learning → Refinement → Optional Execution

It ensures the system always knows what has occurred — and why.

Where to go next