Data Pipeline
Top open-source Data Pipeline AI systems, agents, and MCP tools.
Discover and install verified Data Pipeline AI systems, autonomous agent skills, and Model Context Protocol servers on AIStoreHub. Reusable, security-scanned, and ready for 1-click execution in Cursor, Claude Code, and terminal workflows.
Available AI Systems & Skills (24)
Open in Advanced FilterSelf-hostable workflow orchestrator for teams whose main work isn't orchestration. Declarative YAML over your scripts, SSH commands, containers, etc; keep workflows separate from business logic. One binary, no database, runs on limited H/W resources. Alternative to Airflow / Cron / Job Scheduler.
全球宏观预期概率面板 (AI Skill) — Polymarket+Kalshi 双源 · 5端点 · 10模块 · 零鉴权 | Global-macro-probability panel skill for AI coding assistants (prediction-market data pipeline)
Learn AI Engineering! Learn it. Build it. Ship it for others.
Knowhere extracts, parses, and outputs structured chunks ready for AI Agents and RAG.
Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.
Workflow for AI Agents enables automated conversion of CAD files (such as `.rvt`, `.ifc`, `.dwg`) using command-line converters on a local Windows machine
Academic Research Skills for Claude Code: research → write → review → revise → finalize
Two-pass pipeline for removing AI writing tells from outward-facing text: a surface pass plus a structural pass grounded in the StoryScope study. Packaged as Claude Code Skills. Free community: skool.com/jens-ai-community-1306
Query robotics, drone, and IoT data in plain English through an MCP server, with an intelligent edge data reduction pipeline that keeps only the data that matters.
Publisher-aware full-text PDF fetcher — DOI in, PDF out via OA → publisher TDM APIs → your own institutional proxy. Download end of a paper pipeline; pair with paper-radar (discovery) and paper-review-and-digest (reading).
Influencer discovery & contact enrichment pipeline that runs as a Claude Code skill — 15 channels, stdlib only, appends to a Google Sheet
🎮 AI agent skill that revives shut-down mobile games as fully offline memorial builds — plus a zero-token pipeline that rescues lost game data from screenshots & screen recordings. 停服手游离线化 · 录屏抽帧OCR
SWI is a single-process reference pipeline for security and workflow integrity. Module 00 orchestrates security probing, context checks, PII redaction, drift analysis, memory validation, and audit logging. Modules 01, 04, 08 and 10 provide standalone integrity, encryption, authentication and execution utilities.
Shared Single-file memory layer for all your agents, sub mili-second RAG over text, photo and video on Apple Silicon.. No Server. No API. One File. Pure Swift
An agentic extractor for messaging platforms. Noria receives messages & URLs via chat; scrapes web data to evaluate the content against custom scoring matrices (e.g., jobs, scholarships); and sends a structured summary with a calculated match score directly to the assigned messenger number.
Claude Code and Codex skills for empirical applied-micro research: reproducibility auditing, LLM-pipeline methods, event studies, WRDS, Stata, publication-grade tables
Research pipelines as semantic execution units: each skill declares inputs/outputs, acceptance criteria, and guardrails. Evidence-first methodology prevents hollow writing through structured intermediate artifacts.
Conversion from Excel to structured JSON (tables, shapes, charts) for LLM/RAG pipelines, and autonomous Excel reading/writing by AI agents via CLI and MCP integration.
Production pipeline and Claude Code skill for Vocaloid-style hand-drawn (手書き) lyric music videos: deterministic Canvas2D/WebGL2 animation engine, JIZURA lyric motion, AI illustration, Blender toon 3D, headless rendering to 16:9 and 9:16.
Open-source persistent memory for AI agent pipelines (LangGraph, CrewAI, AutoGen) and Claude. REST API + knowledge graph + autonomous consolidation.
論文學習雷達 — a personal literature-tracking & learning radar: dozens of journal/PubMed feeds → interest-scored → private web triage → flows into your notes. Discovery end of a pipeline: pair with paper-fetch (full-text download) and paper-review-and-digest (appraisal/digest).
Open multilingual construction cost database for AI Agents - 55K+ work items, 27K+ resources, 30 regions. Semantic search via Qdrant vector DB
Workflow for AI Agents enables automated conversion of CAD files (such as `.rvt`, `.ifc`, `.dwg`) using command-line converters on a local Windows machine
Build & Monetize for Data Pipeline
Have you created an autonomous agent, MCP server, or Claude Code skill for Data Pipeline? Publish it to AIStoreHub with dual-currency checkout (USD $ + INR ₹) and earn 85–90% net revenue.