Refactor: Executor Pattern With Self-Configuring Mode Architecture

The Challenge: Untangling the Load Profile System

In the realm of performance testing, particularly within Azure and kperf environments, efficiently managing and executing load profiles is paramount. However, our current load profile system has become a tangled web of limitations, hindering progress and making enhancements a daunting task. The core issues stem from hardcoded mode logic, where the specific behaviors for each execution mode are scattered like digital breadcrumbs across the entire codebase – from the Command Line Interface (CLI) tools to the scheduler and the request generation modules. This fragmentation makes it incredibly difficult to understand, maintain, or modify any single aspect of the system without inadvertently affecting others. Furthermore, there's a tight coupling problem; introducing a new execution mode isn't a simple addition. Instead, it requires a ripple effect of modifications across multiple files and disparate packages, turning what should be a straightforward extension into a significant architectural undertaking. The CLI inflexibility is another major pain point. Flags for crucial parameters like rate, total requests, and duration are hardcoded for specific modes, offering little to no adaptability. This lack of user-friendly configuration means testers are often forced into convoluted workarounds or must accept suboptimal testing scenarios. Most critically, the system suffers from no extensibility. The existing architecture makes it exceptionally difficult to introduce new execution strategies or significantly refactor existing ones without a complete overhaul, stifling innovation and preventing us from adopting more sophisticated testing methodologies. This current state not only slows down development but also increases the risk of introducing bugs and makes the system brittle and hard to evolve.

The Solution: A Clean Executor Pattern with Self-Configuring Modes

To address these deep-rooted architectural limitations, we propose a robust refactoring centered around implementing a clean executor pattern coupled with self-configuring mode types. This new architecture aims to bring clarity, flexibility, and extensibility to our load profile system, making it significantly easier to manage, extend, and use. The core of this solution lies in a few key architectural changes that will fundamentally transform how we handle execution modes and configurations.

Architectural Overhaul: A New Blueprint

At the heart of our proposed solution are several key architectural components designed to decouple logic and enhance flexibility:

• ModeConfig Interface: This is the cornerstone of our new self-configuring system. Each execution mode will now declare its own overridable fields through a GetOverridableFields() method. This allows each mode to explicitly state what parameters can be customized. More importantly, each mode will be responsible for its own validation via a Validate() method, which will include default handling, ensuring that configurations are always sensible. Additionally, modes will be able to configure client options directly through ConfigureClientOptions(), streamlining the setup process. Crucially, our CLI tools will leverage this interface by automatically discovering these overridable fields using a BuildOverridesFromCLI() helper function, eliminating the need for hardcoded flags and providing unparalleled flexibility.
• Executor Pattern: We are introducing a new request/executor package dedicated to housing execution strategies. This package will define an Executor interface, providing a standardized way to implement mode-specific execution logic. Initially, we will introduce two concrete implementations: WeightedRandomExecutor, designed for weighted distribution scenarios with built-in rate limiting, and TimeSeriesExecutor, which will enable replaying load based on time-bucketed audit logs. To manage these executors, an ExecutorFactory will act as a registry, simplifying the process of creating the correct executor instance based on the configured mode.
• Request Builder Adapter: To further enhance decoupling, we are implementing a clean adapter pattern for different request specification types. This will allow us to create request builders specific to each execution mode without unnecessary type conversions. We'll have methods like CreateRequestBuilderFromWeighted() for the weighted-random mode and CreateRequestBuilderFromExact() for the time-series mode, ensuring that the request generation logic is precisely tailored to the execution strategy.
• Mode-Agnostic CLI Tools: A significant benefit of this refactoring is that our CLI tools will become mode-agnostic. They will discover the capabilities of each mode dynamically at runtime, rather than relying on pre-programmed knowledge of specific flags. This means no more hardcoded flags for specific modes! The BuildOverridesFromCLI() helper will automatically extract and apply overrides based on the discovered fields, making the CLI intuitive and adaptable to any new execution mode we introduce.

This comprehensive approach ensures that each component of the system is well-defined, loosely coupled, and highly extensible, paving the way for a more robust and maintainable load profiling solution.

Embracing the Benefits: A Smarter Way to Test

This refactoring isn't just about fixing technical debt; it's about unlocking significant advantages that will streamline our performance testing workflows and enhance our capabilities. The benefits of adopting the executor pattern with self-configuring mode architectures are manifold and directly address the pain points we've been experiencing. Firstly, extensibility is dramatically improved. Adding new execution modes will be as simple as implementing the ModeConfig interface. This means less code duplication, faster iteration cycles, and the ability to quickly adapt to new testing requirements without undertaking massive refactoring efforts. This modularity also leads to a clean separation of concerns. The executor logic, responsible for how requests are dispatched and managed, is now cleanly separated from the scheduler, which orchestrates the overall test execution. This makes each component easier to understand, test, and maintain independently. We also gain significant type safety. With automatic deserialization to the correct concrete type of ModeConfig, we reduce the chances of runtime errors caused by mismatched configurations. The system inherently knows what type of configuration it's dealing with, leading to more reliable test setups. For existing users and configurations, we've ensured backward compatibility. Legacy LoadProfile formats will be automatically migrated to the new structure thanks to custom UnmarshalYAML and UnmarshalJSON methods. This means existing load profiles will continue to work seamlessly without any modifications, ensuring a smooth transition and preventing disruption. Finally, the principle of no coupling is a cornerstone of this design. Our CLI and utility code no longer need to possess intricate knowledge of specific mode types. They interact through the abstract interfaces, making the system inherently more robust and easier to update. If a specific mode's internal logic changes, the CLI and other generic tools remain unaffected, as long as the ModeConfig interface contract is maintained. These benefits collectively pave the way for a more agile, reliable, and powerful performance testing framework.

The Path Forward: A Phased Implementation Plan

To ensure a smooth and manageable transition, this comprehensive refactoring will be executed in four distinct Pull Requests (PRs). This phased approach allows for focused reviews, incremental validation, and minimizes the risk associated with a large, monolithic change. Each PR builds upon the previous one, establishing a solid foundation before moving to more complex logic.

PR1: ModeConfig Interface (The Foundation)

• Objective: Define the core ModeConfig interface and split existing configuration types into distinct, mode-specific structures. This PR also includes adding backward compatibility for the legacy LoadProfile format by implementing custom UnmarshalYAML and UnmarshalJSON methods. Furthermore, test files will be split and organized to align with the new structure.
• Key Files: api/types/mode_config.go, api/types/weighted_random_config.go, api/types/timeseries_config.go
• Rationale: Establishing the ModeConfig interface is the foundational step. It defines the contract for all future execution modes without altering any existing execution logic, making it a low-risk, high-impact change. It sets the stage for all subsequent modifications.
• Status: The branch mode-config-interface is ready for review.

PR2: Executor Pattern (The Core Engine)

• Objective: Implement the executor pattern within the new request/executor/ package. This involves defining the Executor interface and concrete implementations like WeightedRandomExecutor and TimeSeriesExecutor. It also includes implementing the adapter pattern for request builders and refactoring the existing scheduler to leverage these new executors.
• Key Files: request/executor/, request/builders.go, request/schedule.go, request/random.go
• Rationale: This PR contains the core refactoring of the execution logic. By abstracting the execution strategy into distinct executors, we achieve the desired separation and extensibility. This is where the system's new execution paradigm comes to life.
• Dependencies: This PR depends on PR1.

PR3: Mode-Agnostic CLI Tools (User Interface)

• Objective: Update the CLI tools, specifically kperf runner and runkperf, to become mode-agnostic. This involves removing hardcoded flags and utilizing the BuildOverridesFromCLI() helper function to dynamically discover and apply mode-specific configuration overrides at runtime.
• Key Files: cmd/kperf/commands/runner/runner.go, contrib/cmd/runkperf/commands/bench/utils.go
• Rationale: This change decouples the user interface (CLI) from the underlying execution logic. It ensures that the CLI can adapt to any new mode without code changes, providing a consistent and flexible user experience.
• Dependencies: This PR depends on PR1 and PR2.

PR4: Time-Series Benchmark Example (Demonstration)

• Objective: Add a concrete example of a time-series benchmark to demonstrate the new architecture in action. This includes updating example YAML files to reflect the new configuration format and ensuring the example runs correctly.
• Key Files: contrib/cmd/runkperf/commands/bench/timeseries_simple.go
• Rationale: This PR serves as a practical demonstration and validation of the entire new architecture. It provides a clear example for users and developers, showcasing how to leverage the new features and confirming the end-to-end functionality.
• Dependencies: This PR depends on PR1, PR2, and PR3.

This phased approach ensures that each part of the system is robustly developed and reviewed before integration, leading to a higher quality and more stable final product.

Ensuring a Seamless Transition: Backward Compatibility

One of the most critical aspects of this refactoring effort is ensuring a zero-impact transition for our users. We understand that introducing significant architectural changes can often lead to compatibility issues, disrupting existing workflows and requiring extensive updates to user-provided configurations. To preempt this, we have meticulously designed the new system to be backward compatible. This means that all existing LoadProfile formats will continue to work seamlessly without requiring any modifications. We achieve this through custom UnmarshalYAML and UnmarshalJSON methods implemented within the new configuration types. When the system encounters a legacy LoadProfile format, these methods will automatically and transparently migrate it to the new, self-configuring structure. This ensures that existing load profiles, which have been validated and used in production, remain functional and require no user intervention. This commitment to backward compatibility significantly lowers the barrier to adopting the new architecture, allowing teams to benefit from the improvements without the immediate overhead of reconfiguring their tests. It’s a fundamental part of our strategy to make the upgrade process as smooth and painless as possible.

Rigorous Testing: Confidence in Every Change

Confidence in any new system, especially one as critical as our load profiling tool, is built upon a foundation of thorough testing. We have undertaken a comprehensive testing strategy to ensure the robustness and reliability of the refactored architecture. All existing tests have been passed without modification, demonstrating that the core functionality remains intact. Beyond this, we have introduced a suite of additional tests specifically designed to validate the new components and address potential edge cases. These new tests cover several key areas: polymorphic deserialization, ensuring that the system correctly identifies and processes different ModeConfig types; backward compatibility for the legacy format, confirming that the automatic migration works as expected; CLI override extraction, verifying that the mode-agnostic CLI correctly parses and applies overrides; mode-specific configuration validation, making sure each mode's unique settings are handled correctly; and finally, executor pattern functionality, testing the core logic of how requests are executed under different strategies. This multi-layered testing approach provides a high degree of assurance that the new architecture is not only functional but also resilient and dependable. You can find detailed test coverage within the respective PRs.

For more information on Azure services and performance tuning, you can refer to the official Azure documentation. For deeper insights into performance engineering best practices, the Perficient performance engineering resources are highly valuable.