Open Source - MIT

Mako

Real-time data pipelines

Declarative framework for orchestrating data pipelines. Configure your sources, transforms and sinks in YAML — Mako handles the rest.

go install github.com/Stefen-Taime/mako@latest
GitHub

Features

Declarative

Everything is configured in YAML. No code needed for standard pipelines.

Real-time

Native Kafka, PostgreSQL CDC and HTTP streaming support.

WASM Transforms

Go/Rust/TinyGo plugins compiled to WebAssembly for performant transformations.

Observability

Built-in Prometheus metrics, Grafana dashboards and Slack alerts.

Quick Start

# pipeline.yaml
source:
  type: http
  url: https://api.example.com/data
  format: json

transforms:
  - type: field
    operations:
      - rename: { from: "old_name", to: "new_name" }
      - drop: ["unused_field"]

sink:
  type: postgres
  connection: postgres://user:pass@host/db
  table: events
  mode: upsert

Sources

  • • HTTP/REST APIs (pagination, OAuth2, rate limiting)
  • • JSON, CSV, Parquet files (gzip)
  • • Apache Kafka
  • • PostgreSQL CDC (Change Data Capture)
  • • DuckDB (embedded SQL queries)

Sinks

  • • PostgreSQL, Snowflake, BigQuery, ClickHouse
  • • S3, Google Cloud Storage
  • • DuckDB, Kafka, stdout

Transforms

  • • SQL enrichment via DuckDB
  • • WASM plugins (Go/Rust/TinyGo)
  • • Schema validation (Confluent Schema Registry)
  • • Data quality checks
  • • PII masking (SHA-256)
  • • Field operations (rename, drop, cast, flatten, deduplicate)

Orchestration

DAG-based workflow engine with parallel execution and SQL quality gates.

MIT License - Maintained by mcsEdition

godatapipelineskafkayamlwasm

Go 1.21+ required