Algorithm Kit

Algorithm Kit is a comprehensive educational resource providing clean, well-documented implementations of fundamental algorithms in control theory, machine learning, and optimization. Designed for academic instruction and research, this collection offers production-ready Python implementations with detailed explanations, complexity analysis, and practical examples.

Each algorithm is carefully implemented with modern software engineering practices, comprehensive testing, and extensive documentation to serve as both a learning tool and a reliable reference for students and researchers.

Documentation · Source Code · GitHub

Algorithm Families

Explore the comprehensive collection of algorithm implementations organized by domain and application area.

Family	Algorithms	Completion	Status
Control Algorithms	H-Infinity Control, Sliding Mode Control, Robust Control, PID Control, Adaptive Control	0%	Coming Soon
Dynamic Movement Primitives	DMPs with Obstacle Avoidance, Spatially Coupled Bimanual DMPs, Constrained Dynamic Movement Primitives (CDMPs), DMPs for Human-Robot Interaction, Multi-task DMP Learning, Geometry-aware Dynamic Movement Primitives, Online DMP Adaptation, Temporal Dynamic Movement Primitives, DMPs for Manipulation, Basic Dynamic Movement Primitives (DMPs), Probabilistic Movement Primitives (ProMPs), Hierarchical Dynamic Movement Primitives, DMPs for Locomotion, Reinforcement Learning DMPs	0%	Coming Soon
Dynamic Programming	Coin Change Problem, Fibonacci Sequence ✓, 0/1 Knapsack Problem	33%	Coming Soon
Gaussian Process	Deep Gaussian Processes, Multi-Output Gaussian Processes, Sparse Gaussian Processes, GP Classification, GP Optimization, GP Regression	0%	Coming Soon
Hierarchical Reinforcement Learning	Hierarchical Q-Learning, Hierarchical Task Networks (HTNs), Option-Critic, Hierarchical Actor-Critic (HAC), Hierarchical Policy Gradient, Feudal Networks (FuN)	0%	Coming Soon
Model Predictive Control	Distributed MPC, Economic MPC, Robust MPC, Linear MPC, Nonlinear MPC, Model Predictive Control, Learning MPC	0%	Coming Soon
Planning Algorithms	A Search, Breadth-First Search, Depth-First Search, M, Partial Order Planning, GraphPlan, Fast Forward	0%	Coming Soon
Real-time Control	Real-Time PID, Real-Time Adaptive Control, Real-Time MPC, Real-Time Control	0%	Coming Soon
Reinforcement Learning	SARSA, Actor-Critic, Policy Gradient, Q-Learning, Proximal Policy Optimization (PPO), Deep Q-Network (DQN)	0%	Coming Soon

Status legend: Code available · Coming Soon

Family Overviews

Each algorithm family represents a distinct computational paradigm with specific theoretical foundations and practical applications. Click through the families below to explore the algorithms, their mathematical foundations, and implementation details.

Control Algorithms Dynamic Movement Primitives Dynamic Programming Gaussian Process Hierarchical Reinforcement Learning Model Predictive Control Planning Algorithms Real-time Control Reinforcement Learning

Control Algorithms

Control algorithms provide methods to regulate system behavior, maintain desired outputs, and ensure stability under various operating conditions.

Completion: 0% (0 of 5 algorithms complete)

Algorithms:

• H-Infinity Control - Robust control design methodology that minimizes the worst-case performance of a system under bounded disturbances and uncertainties.
• Sliding Mode Control - Robust control strategy that forces the system state to reach and remain on a predefined sliding surface, regardless of parameter uncertainties and external disturbances.
• Robust Control - Comprehensive control design methodology that ensures system stability and performance despite model uncertainties, parameter variations, and external disturbances.
• PID Control - Fundamental feedback control algorithm combining proportional, integral, and derivative actions to achieve desired system behavior.
• Adaptive Control - Control strategy that automatically adjusts controller parameters based on real-time system identification and performance evaluation.

Dynamic Movement Primitives

Dynamic Movement Primitives provide a framework for learning, representing, and reproducing complex motor behaviors in robotics and control systems.

Completion: 0% (0 of 14 algorithms complete)

Algorithms:

• DMPs with Obstacle Avoidance - DMPs enhanced with real-time obstacle avoidance capabilities using repulsive forces and safe navigation in cluttered environments.
• Spatially Coupled Bimanual DMPs - DMPs for coordinated dual-arm movements with spatial coupling between arms for synchronized manipulation tasks and hand-eye coordination.
• Constrained Dynamic Movement Primitives (CDMPs) - DMPs with safety constraints and operational requirements that ensure movements comply with safety limits and operational constraints.
• DMPs for Human-Robot Interaction - DMPs specialized for human-robot interaction including imitation learning, collaborative tasks, and social robot behaviors.
• Multi-task DMP Learning - DMPs that learn from multiple demonstrations across different tasks, enabling task generalization and cross-task knowledge transfer.
• Geometry-aware Dynamic Movement Primitives - DMPs that operate with symmetric positive definite matrices to handle stiffness and damping matrices for impedance control applications.
• Online DMP Adaptation - DMPs with real-time parameter updates, continuous learning from feedback, and adaptive behavior modification during execution.
• Temporal Dynamic Movement Primitives - DMPs that generate time-based movements with rhythmic pattern learning, beat and tempo adaptation for temporal movement generation.
• DMPs for Manipulation - DMPs specialized for robotic manipulation tasks including grasping movements, assembly tasks, and tool use behaviors.
• Basic Dynamic Movement Primitives (DMPs) - Fundamental DMP framework for learning and reproducing point-to-point and rhythmic movements with temporal and spatial scaling.
• Probabilistic Movement Primitives (ProMPs) - Probabilistic extension of DMPs that captures movement variability and generates movement distributions from multiple demonstrations.
• Hierarchical Dynamic Movement Primitives - DMPs organized in hierarchical structures for multi-level movement decomposition, complex behavior composition, and task hierarchy learning.
• DMPs for Locomotion - DMPs specialized for walking pattern generation, gait adaptation, and terrain-aware movement in legged robots and humanoid systems.
• Reinforcement Learning DMPs - DMPs enhanced with reinforcement learning for parameter optimization, reward-driven learning, and policy gradient methods for movement refinement.

Dynamic Programming

Dynamic Programming solves complex problems by breaking them into overlapping subproblems with optimal substructure.

Completion: 33% (1 of 3 algorithms complete)

Algorithms:

• Coin Change Problem - Find the minimum number of coins needed to make a given amount using dynamic programming.
• Fibonacci Sequence ✓ - Classic sequence where each number is the sum of the two preceding ones, demonstrating recursion, memoization, and dynamic programming.
• 0/1 Knapsack Problem - Optimize value within weight constraints using dynamic programming for resource allocation.

Gaussian Process

Gaussian Process algorithms provide probabilistic machine learning methods for regression, classification, and optimization with uncertainty quantification.

Completion: 0% (0 of 6 algorithms complete)

Algorithms:

• Deep Gaussian Processes - Hierarchical extension of Gaussian processes that stacks multiple GP layers to model complex, non-stationary functions with improved scalability.
• Multi-Output Gaussian Processes - Extension of Gaussian processes to handle multiple correlated outputs simultaneously, enabling joint modeling and knowledge transfer between tasks.
• Sparse Gaussian Processes - Scalable Gaussian process methods using inducing points to reduce computational complexity from O(n³) to O(nm²) for large datasets.
• GP Classification - Probabilistic classification method using Gaussian processes with sigmoid link functions for binary and multi-class problems.
• GP Optimization - Global optimization method using Gaussian processes to efficiently explore and exploit the objective function with uncertainty-aware acquisition functions.
• GP Regression - Probabilistic regression method that provides both predictions and uncertainty estimates using Gaussian processes.

Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning decomposes complex tasks into simpler subtasks using temporal abstraction and multi-level decision making.

Completion: 0% (0 of 6 algorithms complete)

Algorithms:

• Hierarchical Q-Learning - Extends traditional Q-Learning to handle temporal abstraction and hierarchical task decomposition with multi-level Q-functions.
• Hierarchical Task Networks (HTNs) - A hierarchical reinforcement learning approach that decomposes complex tasks into hierarchical structures of subtasks for planning and execution.
• Option-Critic - A hierarchical reinforcement learning algorithm that learns options (temporally extended actions) end-to-end using policy gradient methods.
• Hierarchical Actor-Critic (HAC) - An advanced hierarchical reinforcement learning algorithm that extends the actor-critic framework with temporal abstraction and hierarchical structure.
• Hierarchical Policy Gradient - Extends traditional policy gradient methods to handle temporal abstraction and hierarchical task decomposition with multi-level policies.
• Feudal Networks (FuN) - A hierarchical reinforcement learning algorithm that implements a manager-worker architecture for temporal abstraction and goal-based learning.

Model Predictive Control

Model Predictive Control optimizes control actions by solving constrained optimization problems over a prediction horizon.

Completion: 0% (0 of 7 algorithms complete)

Algorithms:

• Distributed MPC - Model Predictive Control for large-scale systems using distributed optimization and coordination between multiple local controllers to achieve global objectives.
• Economic MPC - Model Predictive Control that optimizes economic objectives rather than tracking performance, focusing on profit maximization and cost minimization in process industries.
• Robust MPC - Model Predictive Control that handles model uncertainty and disturbances through robust optimization techniques to ensure constraint satisfaction and stability.
• Linear MPC - Model Predictive Control for linear time-invariant systems formulated as a Quadratic Programming problem with efficient real-time solution.
• Nonlinear MPC - Model Predictive Control for nonlinear systems using Sequential Quadratic Programming to handle complex dynamics and constraints.
• Model Predictive Control - Advanced control strategy that uses system models to predict future behavior and optimize control actions over a finite horizon while handling constraints.
• Learning MPC - Model Predictive Control that learns system dynamics and improves performance through data-driven approaches, combining machine learning with predictive control.

Planning Algorithms

Planning algorithms solve sequential decision problems by finding optimal sequences of actions to achieve goals from initial states.

Completion: 0% (0 of 7 algorithms complete)

Algorithms:

• A* Search - Optimal pathfinding algorithm that uses heuristics to efficiently find the shortest path from start to goal in weighted graphs.
• Breadth-First Search - Graph traversal algorithm that explores all nodes at the current depth level before moving to the next level, guaranteeing shortest path in unweighted graphs.
• Depth-First Search - Graph traversal algorithm that explores as far as possible along each branch before backtracking, using stack-based or recursive implementation.
• M* - Multi-agent pathfinding algorithm that finds collision-free paths for multiple agents by resolving conflicts and coordinating their movements.
• Partial Order Planning - Planning algorithm that maintains partial ordering of actions, allowing flexible execution order and handling of uncertain environments.
• GraphPlan - Classical planning algorithm that builds a planning graph to find parallel action sequences for achieving goals from initial states.
• Fast Forward - Heuristic planning algorithm that uses relaxed planning graphs to efficiently find solutions to STRIPS planning problems.

Real-time Control

Real-time control algorithms provide deterministic, time-critical control solutions for systems requiring guaranteed response times and predictable behavior.

Completion: 0% (0 of 4 algorithms complete)

Algorithms:

• Real-Time PID - Proportional-Integral-Derivative control algorithm optimized for real-time systems with guaranteed execution time and deterministic behavior.
• Real-Time Adaptive Control - Adaptive control algorithms that automatically adjust controller parameters in real-time to maintain performance under changing conditions.
• Real-Time MPC - Model Predictive Control adapted for real-time systems with timing constraints, providing optimal control with guaranteed response times.
• Real-Time Control - Control algorithms designed to operate within strict timing constraints, providing deterministic and predictable behavior for time-critical systems.

Reinforcement Learning

Reinforcement Learning enables agents to learn optimal behavior through interaction with an environment using rewards and penalties.

Completion: 0% (0 of 6 algorithms complete)

Algorithms:

• SARSA - An on-policy temporal difference learning algorithm that learns action-value functions by following the current policy.
• Actor-Critic - A hybrid reinforcement learning algorithm that combines policy gradient methods with value function approximation for improved learning efficiency.
• Policy Gradient - A policy-based reinforcement learning algorithm that directly optimizes the policy using gradient ascent on expected returns.
• Q-Learning - A model-free reinforcement learning algorithm that learns optimal action-value functions through temporal difference learning.
• Proximal Policy Optimization (PPO) - A state-of-the-art policy gradient algorithm that uses clipped objective to ensure stable policy updates with improved sample efficiency.
• Deep Q-Network (DQN) - A deep reinforcement learning algorithm that uses neural networks to approximate Q-functions for high-dimensional state spaces.

Getting Started

Installation and Setup

To begin using Algorithm Kit in your research or coursework:

• Install: uv pip install -e . (editable development install)
• Verify: just test (run the comprehensive test suite)
• Quality Check: just lint (ensure code quality standards)

Command LinePython

# Install Algorithm Kit in development mode
uv pip install -e .

# Verify all implementations with comprehensive tests
just test

# Ensure code quality and style compliance
just lint

# Import the package
import algokit

# Your algorithm implementations here
print("Algorithm Kit is ready!")

Academic Use

This resource is designed for educational and research purposes. Each algorithm includes theoretical background, complexity analysis, and practical implementation details suitable for coursework and research projects.

Key Features

• Academic Quality: Rigorous implementations with theoretical foundations and complexity analysis
• Educational Focus: Comprehensive documentation designed for learning and teaching
• Research Ready: Production-quality code suitable for academic research and publication
• Comprehensive Testing: Extensive test suites ensuring correctness and reliability
• Type Safety: Full type annotations for better code understanding and IDE support
• Modular Design: Clean architecture enabling easy extension and customization

Implementation Standards

• Modern Python: Built with Python 3.12+ and contemporary best practices
• Rigorous Testing: pytest with comprehensive coverage ensuring algorithm correctness
• Code Quality: Automated linting and formatting for consistent, readable implementations
• Type Safety: Complete type annotations for better code understanding and IDE support
• Documentation: Professional documentation with mathematical formulations and examples
• Version Control: Git-based workflow with automated quality assurance

Development

For more detailed information about the project architecture, development guidelines, and quality standards, please refer to the project documentation in the main repository.

API Reference

Spotted an issue? Edit this page.