Enterprise IoT Platform
Multi-Tenant Workflow Automation Infrastructure
Kubernetes-orchestrated workflow engines with full observability
50+
IoT Subflows
10+
Dashboard Charts
7+
Hypertables
1000+
Instances
The Problem
IoT organizations managing thousands of devices need workflow automation that scales. Standard tools lack enterprise features: no database persistence for high availability, no distributed tracing for debugging, no event streaming integration, and no multi-tenant isolation. Manual Kubernetes provisioning doesn't scale beyond dozens of instances.
The Solution
Built a three-component platform: (1) Extended Node-RED with 15 custom enterprise nodes, PostgreSQL persistence replacing file storage, Kafka event streaming, and full OpenTelemetry tracing. (2) Two Go-based Kubernetes controllers that automatically provision complete workflow environments (8 resources each) from database configuration with circuit breaker resilience. (3) TypeScript/Fastify gateway providing unified authentication, WebSocket proxying, and real-time cache invalidation via PostgreSQL LISTEN/NOTIFY. **Lesson learned**: The first version of the Kubernetes controller used a simple polling loop that caused race conditions during rapid deployments. Implementing proper leader election, exponential backoff, and circuit breakers transformed it from "works most of the time" to production-grade reliability with 1-second reconciliation guarantees.
Tech Stack
Frontend
Workflow Engine
Backend
Kubernetes
Gateway
Observability
My Role: Senior Data Architect
- Architected entire platform from workflow engine to Kubernetes orchestration
- Built 10+ React dashboard components with custom hooks and auto-refresh
- Designed Generic Timeseries API (Go/TimescaleDB) replacing 5+ specific endpoints
- Implemented 7+ TimescaleDB hypertables with compression and retention policies
- Built 15 custom Node-RED nodes with automatic OpenTelemetry tracing (BaseNode pattern)
- Created two Go Kubernetes controllers with circuit breaker and retry logic
- Built TypeScript gateway with browser fingerprinting and real-time cache invalidation
- Integrated full OpenTelemetry suite (traces, metrics, logs) across all components
Platform Components
Fleet Analytics Dashboard
Real-time IoT fleet visualization with 10+ chart components, custom React hooks, and auto-refresh capabilities for monitoring device health and workflows.
- 10+ visualization components (Heartbeat, Workflows, Errors, Network, Reports)
- Custom hooks: useHeartbeatStats, useMetricsData, useWorkflows with auto-refresh
- Material-UI + Recharts for responsive data visualization
- IntervalSelector and DateRangePicker with smart formatting
Generic Timeseries API
Go/TimescaleDB backend supporting dynamic time-series queries without backend changes for each new visualization - reduced dashboard development from weeks to hours.
- Single endpoint replaced 5+ specific endpoints
- Advanced aggregations: percentiles (p50, p99), histograms, stddev
- Pivot with top-N for high-cardinality fields
- 7+ hypertables with automatic compression and retention
Workflow Engine
Extended Node-RED with 15 custom enterprise nodes, 50+ production-ready subflows, and a unique visual development → locked runtime pattern for rapid iteration with production stability.
- 50+ subflows covering complete IoT lifecycle (device events, gateway commands, notifications)
- Visual development → locked runtime: edit externally, deploy as immutable nodes
- 15 custom nodes with BaseNode pattern for automatic OpenTelemetry tracing
- PostgreSQL persistence + Kafka event streaming for enterprise scale
Kubernetes Controllers
Two Go-based controllers enabling database-as-single-source-of-truth for Kubernetes deployments. Add a row to PostgreSQL, get a fully provisioned workflow environment automatically.
- Database is single source of truth - no kubectl or manifests needed
- Automatic provisioning of 8 K8s resources per tenant from DB records
- 1-second reconciliation cycle syncs desired vs actual state
- Circuit breaker resilience for graceful failure handling
Key Differentiators
End-to-End Platform: From workflow engine to Kubernetes orchestration in one solution
Database-Driven Architecture: PostgreSQL-based reconciliation enables external system integration
Full Observability: OpenTelemetry tracing across Node-RED, controllers, and gateway
Enterprise Resilience: Circuit breakers, retry logic, health monitoring throughout
Multi-Tenant Isolation: Namespace-level separation with per-instance secrets and network policies
Generic Timeseries API: Single endpoint handles any time-series query vs. endpoint-per-chart
Interested in a Similar Project?
See how I can help with your IoT platform
Other Projects
Zeitgaist
Cross-Lingual Social Intelligence
Synthesizes insights from 6 platforms in seconds vs hours of manual monitoring, with cross-lingual search capabilities.
Foretale
Real-Time NLP & Multi-Source Data Fusion Platform
Platform processes 100K+ daily data points across 7+ sources with sub-second latency, running 24/7 with ML-powered sentiment analysis and OCR extraction.