|
Thread System 0.3.1
High-performance C++20 thread pool with work stealing and DAG scheduling
|
DAG-based job scheduler with dependency management. More...
#include <dag_scheduler.h>

Public Member Functions | |
| dag_scheduler (std::shared_ptr< thread_pool > pool, dag_config config={}) | |
| Constructs a DAG scheduler with a thread pool. | |
| ~dag_scheduler () | |
| Destructor - cancels any pending jobs. | |
| dag_scheduler (const dag_scheduler &)=delete | |
| auto | operator= (const dag_scheduler &) -> dag_scheduler &=delete |
| dag_scheduler (dag_scheduler &&) noexcept | |
| auto | operator= (dag_scheduler &&) noexcept -> dag_scheduler & |
| auto | add_job (std::unique_ptr< dag_job > j) -> job_id |
| Adds a job to the DAG. | |
| auto | add_job (dag_job_builder &&builder) -> job_id |
| Adds a job using a builder. | |
| auto | add_dependency (job_id dependent, job_id dependency) -> common::VoidResult |
| Adds a dependency between jobs. | |
| auto | remove_job (job_id id) -> common::VoidResult |
| Removes a job from the DAG (only if not yet started) | |
| auto | execute_all () -> std::future< common::VoidResult > |
| Executes all jobs in dependency order. | |
| auto | execute (job_id target) -> std::future< common::VoidResult > |
| Executes a specific job and its dependencies. | |
| auto | cancel_all () -> void |
| Cancels all pending jobs. | |
| auto | wait () -> common::VoidResult |
| Waits for all jobs to complete. | |
| auto | reset () -> common::VoidResult |
| Resets the scheduler for reuse. | |
| auto | get_job_info (job_id id) const -> std::optional< dag_job_info > |
| Gets information about a specific job. | |
| auto | get_all_jobs () const -> std::vector< dag_job_info > |
| Gets information about all jobs. | |
| auto | get_jobs_in_state (dag_job_state state) const -> std::vector< dag_job_info > |
| Gets jobs in a specific state. | |
| auto | get_ready_jobs () const -> std::vector< job_id > |
| Gets IDs of ready jobs (dependencies satisfied) | |
| auto | has_cycles () const -> bool |
| Checks if the DAG has cycles. | |
| auto | get_execution_order () const -> std::vector< job_id > |
| Gets topological execution order. | |
| template<typename T > | |
| auto | get_result (job_id id) const -> const T & |
| Gets the result from a completed job. | |
| auto | to_dot () const -> std::string |
| Exports the DAG as DOT format (Graphviz) | |
| auto | to_json () const -> std::string |
| Exports the DAG as JSON format. | |
| auto | get_stats () const -> dag_stats |
| Gets execution statistics. | |
| auto | get_config () const -> const dag_config & |
| Gets the configuration. | |
Private Member Functions | |
| auto | on_job_completed (job_id id) -> void |
| Called when a job completes successfully. | |
| auto | on_job_failed (job_id id, const std::string &error) -> void |
| Called when a job fails. | |
| auto | schedule_ready_jobs () -> void |
| Schedules ready jobs for execution. | |
| auto | execute_job (job_id id) -> void |
| Executes a single job. | |
| auto | topological_sort () const -> std::vector< job_id > |
| Performs topological sort. | |
| auto | detect_cycle () const -> bool |
| Detects cycles using DFS. | |
| auto | are_dependencies_satisfied (job_id id) const -> bool |
| Checks if a job's dependencies are all satisfied. | |
| auto | skip_dependents (job_id failed_id) -> void |
| Marks dependents as skipped due to dependency failure. | |
| auto | cancel_dependents (job_id failed_id) -> void |
| Cancels dependents due to dependency failure. | |
| auto | is_execution_complete () const -> bool |
| Checks if execution is complete. | |
Static Private Member Functions | |
| static auto | get_state_color (dag_job_state state) -> std::string |
| Gets the state color for DOT visualization. | |
Private Attributes | |
| std::shared_ptr< thread_pool > | pool_ |
| Thread pool for job execution. | |
| dag_config | config_ |
| Configuration. | |
| std::unordered_map< job_id, std::unique_ptr< dag_job > > | jobs_ |
| Job storage (job_id -> dag_job) | |
| std::unordered_map< job_id, std::vector< job_id > > | dependencies_ |
| Dependency graph (job -> jobs it depends on) | |
| std::unordered_map< job_id, std::vector< job_id > > | dependents_ |
| Reverse dependency graph (job -> jobs that depend on it) | |
| std::shared_mutex | mutex_ |
| Mutex for thread-safe access. | |
| std::condition_variable_any | completion_cv_ |
| Condition variable for waiting on completion. | |
| std::atomic< bool > | executing_ {false} |
| Flag indicating execution is in progress. | |
| std::atomic< bool > | cancelled_ {false} |
| Flag indicating cancellation was requested. | |
| std::atomic< std::size_t > | running_count_ {0} |
| Number of jobs currently running. | |
| std::atomic< std::size_t > | active_callbacks_ {0} |
| Number of active callbacks (for safe destruction) | |
| std::chrono::steady_clock::time_point | execution_start_time_ |
| Execution start time. | |
| std::optional< common::error_info > | first_error_ |
| First error encountered during execution. | |
| std::unordered_map< job_id, std::size_t > | retry_counts_ |
| Retry count per job. | |
DAG-based job scheduler with dependency management.
The dag_scheduler manages jobs with dependencies, ensuring they execute in the correct order. Jobs are represented as a Directed Acyclic Graph (DAG) where edges represent dependencies.
All public methods are thread-safe and can be called from any thread. Internal state is protected by a shared_mutex for optimal read performance.
Definition at line 78 of file dag_scheduler.h.
|
explicit |
Constructs a DAG scheduler with a thread pool.
| pool | The thread pool to use for job execution |
| config | Optional configuration (default: dag_config{}) |
Definition at line 18 of file dag_scheduler.cpp.
| kcenon::thread::dag_scheduler::~dag_scheduler | ( | ) |
Destructor - cancels any pending jobs.
Definition at line 24 of file dag_scheduler.cpp.
References active_callbacks_, and cancel_all().

|
delete |
|
noexcept |
Definition at line 36 of file dag_scheduler.cpp.
|
nodiscard |
Adds a dependency between jobs.
| dependent | The job that depends on another |
| dependency | The job being depended on |
Thread Safety: Thread-safe (acquires exclusive lock)
Definition at line 109 of file dag_scheduler.cpp.
References kcenon::thread::invalid_argument, kcenon::thread::job_invalid, and kcenon::thread::make_error_result().

|
nodiscard |
Adds a job using a builder.
| builder | The job builder |
Thread Safety: Thread-safe (acquires exclusive lock)
Definition at line 104 of file dag_scheduler.cpp.
Adds a job to the DAG.
| j | The job to add |
Thread Safety: Thread-safe (acquires exclusive lock)
Definition at line 74 of file dag_scheduler.cpp.
References kcenon::thread::INVALID_JOB_ID.
|
nodiscardprivate |
Checks if a job's dependencies are all satisfied.
| id | The job ID to check |
Definition at line 1026 of file dag_scheduler.cpp.
References kcenon::thread::completed.
Referenced by get_ready_jobs().

| auto kcenon::thread::dag_scheduler::cancel_all | ( | ) | -> void |
Cancels all pending jobs.
Jobs that are already running will complete, but no new jobs will start.
Thread Safety: Thread-safe
Definition at line 322 of file dag_scheduler.cpp.
References kcenon::thread::cancelled, kcenon::thread::pending, and kcenon::thread::ready.
Referenced by ~dag_scheduler().

|
private |
Cancels dependents due to dependency failure.
| failed_id | The failed job ID |
Definition at line 1097 of file dag_scheduler.cpp.
References kcenon::thread::cancelled, kcenon::thread::pending, and kcenon::thread::ready.
|
nodiscardprivate |
Detects cycles using DFS.
Definition at line 982 of file dag_scheduler.cpp.
References dependencies_, and jobs_.
Referenced by has_cycles().

|
nodiscard |
Executes a specific job and its dependencies.
| target | The target job to execute |
Thread Safety: Thread-safe
Definition at line 230 of file dag_scheduler.cpp.
References kcenon::thread::job_invalid, kcenon::thread::make_error_result(), kcenon::thread::pending, kcenon::thread::ready, and kcenon::thread::thread_already_running.

|
nodiscard |
Executes all jobs in dependency order.
Thread Safety: Thread-safe
Definition at line 185 of file dag_scheduler.cpp.
References kcenon::thread::make_error_result(), kcenon::thread::pending, kcenon::thread::ready, and kcenon::thread::thread_already_running.

|
private |
Executes a single job.
| id | The job ID to execute |
Definition at line 868 of file dag_scheduler.cpp.
References kcenon::thread::result< T >::is_ok(), kcenon::thread::job_invalid, kcenon::thread::make_error_result(), kcenon::thread::ready, and kcenon::thread::running.

|
nodiscard |
Gets information about all jobs.
Thread Safety: Thread-safe (acquires shared lock)
Definition at line 405 of file dag_scheduler.cpp.
References dependents_, kcenon::thread::info, jobs_, and mutex_.
|
inlinenodiscard |
Gets the configuration.
Definition at line 306 of file dag_scheduler.h.
References config_.
|
nodiscard |
Gets topological execution order.
Thread Safety: Thread-safe (acquires shared lock)
Definition at line 476 of file dag_scheduler.cpp.
References mutex_, and topological_sort().

|
nodiscard |
Gets information about a specific job.
| id | The job ID |
Thread Safety: Thread-safe (acquires shared lock)
Definition at line 383 of file dag_scheduler.cpp.
References kcenon::thread::info.
|
nodiscard |
Gets jobs in a specific state.
| state | The state to filter by |
Thread Safety: Thread-safe (acquires shared lock)
Definition at line 426 of file dag_scheduler.cpp.
References kcenon::thread::info.
|
nodiscard |
Gets IDs of ready jobs (dependencies satisfied)
Thread Safety: Thread-safe (acquires shared lock)
Definition at line 449 of file dag_scheduler.cpp.
References are_dependencies_satisfied(), jobs_, mutex_, kcenon::thread::pending, and kcenon::thread::ready.

|
inlinenodiscard |
Gets the result from a completed job.
| T | The expected result type |
| id | The job ID |
| std::runtime_error | if job not found or not completed |
| std::bad_any_cast | if type doesn't match |
Thread Safety: Thread-safe (acquires shared lock)
Definition at line 255 of file dag_scheduler.h.
References kcenon::thread::completed, jobs_, and mutex_.
|
staticnodiscardprivate |
Gets the state color for DOT visualization.
| state | The job state |
Definition at line 1144 of file dag_scheduler.cpp.
References kcenon::thread::cancelled, kcenon::thread::completed, kcenon::thread::failed, kcenon::thread::pending, kcenon::thread::ready, kcenon::thread::running, and kcenon::thread::skipped.
Referenced by to_dot().

|
nodiscard |
Gets execution statistics.
Thread Safety: Thread-safe (acquires shared lock)
Definition at line 600 of file dag_scheduler.cpp.
References kcenon::thread::cancelled, kcenon::thread::dag_stats::cancelled_jobs, kcenon::thread::completed, kcenon::thread::dag_stats::completed_jobs, executing_, execution_start_time_, kcenon::thread::failed, kcenon::thread::dag_stats::failed_jobs, jobs_, kcenon::thread::pending, kcenon::thread::dag_stats::pending_jobs, kcenon::thread::ready, kcenon::thread::running, kcenon::thread::dag_stats::running_jobs, kcenon::thread::skipped, kcenon::thread::dag_stats::skipped_jobs, kcenon::thread::dag_stats::total_execution_time, and kcenon::thread::dag_stats::total_jobs.
Referenced by to_json().

|
nodiscard |
Checks if the DAG has cycles.
Thread Safety: Thread-safe (acquires shared lock)
Definition at line 470 of file dag_scheduler.cpp.
References detect_cycle(), and mutex_.

|
nodiscardprivate |
Checks if execution is complete.
Definition at line 1159 of file dag_scheduler.cpp.
References jobs_, kcenon::thread::pending, kcenon::thread::ready, and kcenon::thread::running.
|
private |
Called when a job completes successfully.
| id | The completed job ID |
Definition at line 635 of file dag_scheduler.cpp.
References kcenon::thread::completed, kcenon::thread::pending, and kcenon::thread::ready.
|
private |
Called when a job fails.
| id | The failed job ID |
| error | The error message |
Definition at line 692 of file dag_scheduler.cpp.
References kcenon::thread::completed, kcenon::thread::continue_others, kcenon::thread::fail_fast, kcenon::thread::failed, kcenon::thread::fallback, kcenon::thread::result< T >::is_ok(), kcenon::thread::job_execution_failed, kcenon::thread::ready, kcenon::thread::retry, kcenon::thread::running, and kcenon::thread::to_error_info().

|
delete |
|
noexcept |
Definition at line 51 of file dag_scheduler.cpp.
|
nodiscard |
Removes a job from the DAG (only if not yet started)
| id | The job ID to remove |
Thread Safety: Thread-safe (acquires exclusive lock)
Definition at line 145 of file dag_scheduler.cpp.
References kcenon::thread::job_invalid, kcenon::thread::make_error_result(), and kcenon::thread::running.

|
nodiscard |
Resets the scheduler for reuse.
Clears all jobs and resets state. Cannot be called while jobs are running.
Thread Safety: Thread-safe (acquires exclusive lock)
Definition at line 359 of file dag_scheduler.cpp.
References kcenon::thread::make_error_result(), and kcenon::thread::thread_already_running.

|
private |
Schedules ready jobs for execution.
Definition at line 821 of file dag_scheduler.cpp.
References kcenon::thread::ready, and kcenon::thread::running.
|
private |
Marks dependents as skipped due to dependency failure.
| failed_id | The failed job ID |
Definition at line 1050 of file dag_scheduler.cpp.
References kcenon::thread::pending, kcenon::thread::ready, and kcenon::thread::skipped.
|
nodiscard |
Exports the DAG as DOT format (Graphviz)
Thread Safety: Thread-safe (acquires shared lock)
Definition at line 486 of file dag_scheduler.cpp.
References kcenon::thread::dag_job_state_to_string(), dependencies_, kcenon::thread::job::get_name(), get_state_color(), jobs_, and mutex_.

|
nodiscard |
Exports the DAG as JSON format.
Thread Safety: Thread-safe (acquires shared lock)
Definition at line 520 of file dag_scheduler.cpp.
References kcenon::thread::dag_job_state_to_string(), dependents_, get_stats(), kcenon::thread::info, jobs_, and mutex_.

|
nodiscardprivate |
Performs topological sort.
Definition at line 925 of file dag_scheduler.cpp.
References dependencies_, dependents_, and jobs_.
Referenced by get_execution_order().

|
nodiscard |
Waits for all jobs to complete.
Thread Safety: Thread-safe
Definition at line 343 of file dag_scheduler.cpp.
|
private |
Number of active callbacks (for safe destruction)
Tracks callbacks that may access members after releasing the mutex. The destructor waits for this to reach 0 before destroying members.
Definition at line 365 of file dag_scheduler.h.
Referenced by ~dag_scheduler().
|
private |
Flag indicating cancellation was requested.
Definition at line 352 of file dag_scheduler.h.
|
private |
Condition variable for waiting on completion.
Definition at line 342 of file dag_scheduler.h.
|
private |
|
private |
Dependency graph (job -> jobs it depends on)
Definition at line 327 of file dag_scheduler.h.
Referenced by detect_cycle(), to_dot(), and topological_sort().
Reverse dependency graph (job -> jobs that depend on it)
Definition at line 332 of file dag_scheduler.h.
Referenced by get_all_jobs(), to_json(), and topological_sort().
|
private |
Flag indicating execution is in progress.
Definition at line 347 of file dag_scheduler.h.
Referenced by get_stats().
|
private |
|
private |
First error encountered during execution.
Definition at line 375 of file dag_scheduler.h.
Job storage (job_id -> dag_job)
Definition at line 322 of file dag_scheduler.h.
Referenced by detect_cycle(), get_all_jobs(), get_ready_jobs(), get_result(), get_stats(), is_execution_complete(), to_dot(), to_json(), and topological_sort().
|
mutableprivate |
Mutex for thread-safe access.
Definition at line 337 of file dag_scheduler.h.
Referenced by get_all_jobs(), get_execution_order(), get_ready_jobs(), get_result(), has_cycles(), to_dot(), and to_json().
|
private |
Thread pool for job execution.
Definition at line 312 of file dag_scheduler.h.
|
private |
Retry count per job.
Definition at line 380 of file dag_scheduler.h.
|
private |