Data volumes are exploding, IT landscapes and user behaviour are changing. What does this mean for IT workload management in the data centre? – Below is (a) a brief outline of the development of workload automation (WLA) software and (b) the most important trends around WLA software.
Brief outline of the development of workload automation Software
The market for workload automation software has its origins in the 1970s, as so-called job scheduling on IBM mainframes. The first WLA tools were not very complex, the execution of “jobs” was essentially date and time based.
In the 1980s and 1990s, more advanced scheduling functions were added, including event-based triggers for starting jobs. And as midrange and X86-based systems entered IT production, WLA software was adapted to them. In addition, WLA and Managed File Transfer (MFT) software grew together, as many jobs start and end with data transfers.
In 2005, Gartner coined the term workload automation. A few years later, when Gartner considered this very market to be mature, the analyst firm finally discontinued the Magic Quadrant for this software category (2013).
However, cloud computing and Big Data reawakened interest in this software category. Cloud computing offered flexibility in job execution, but also brought a new level of complexity. Big Data presented significant challenges to ETL: massive file transfers, intensive pre-processing and complex analysis and reporting. WLA adapted and started integrating more extensively with the applications being run to better handle the growing complexity.
Other trends also impacted WLA, including the growth of SLA management and the use of predictive analytics. Self-servicing (or: federated use) across IT was also on the rise, and for WLA this meant, among other things, dashboards for a wider group of users. In addition, the DevOps trend soon had an impact on WLA: Jobs-as-code allowed developers to better define application planning requirements earlier in the development cycle.
WLA becomes Service Orchestration & Automation Platform (SOAP)
In 2019, Gartner launched the Service Orchestration & Automation Platform (SOAP) market report. SOAP solutions are a continuous evolution and transformation of traditional workload automation (WLA) tools to orchestrate and manage applications and infrastructures in an enterprise hybrid IT environment. Gartner‘s definition reflects these same characteristics for high integration into complex workflows and hybrid IT landscapes (see Gartner Reports on SOAP):
Service orchestration and automation platforms (SOAPs) enable infrastructure and operations (I&O) leaders to design and implement business services through a combination of workflow orchestration, workload automation and resource provisioning across an organization’s hybrid digital infrastructure. SOAPs provide a unified administration console and an orchestration engine to manage data pipelines and enable event-driven application workflows. Some tools expose APIs and include mobile apps for scheduling batch processes, monitoring task statuses and alerting users when new events are triggered.
SOAPs expand the role of traditional workload automation by adapting to broader use cases and cloud-native infrastructure and application architecture. These tools complement and integrate with DevOps toolchains to provide customer-focused agility, in addition to cost savings, operational efficiency and process standardization.
The analyst firm Enterprise Management Associates (EMA) (Website www.enterprisemanagement.com) makes the evolution of WLA software to SOAP easy to understand with the following graphic:
The WLA / SOAP market: Status Quo
Let us quantify the relevance of WLA with some figures. According to a study by the analyst firm EMA, the market is worth around USD 3 billion, with a growing trend (compare Global Workload Automation Market Size and Forecast 2022 to 2027). Depending on the premises, growth in the foreseeable future will be higher or lower; the EMA forecast considers two scenarios: The status quo forecast assumes a continuation of the trends in big data pipelines, digital transformation and cloudification. The second scenario assumes a growing automation dynamic. The following graph shows the forecast market development for both scenarios:
And what share of the IT workload is controlled by WLA in companies? (Note: In the enterprise segment of companies where the IT workload reaches a critical limit and software-based orchestration is required)? – According to a data survey by EMA of relevant companies, the WLA software controls about 40% to 45% of IT workloads, tasks and activities (these figures are from 2019; EMA estimates that this share is probably even higher today). For isolated organisations, this share is up to 80%.
Current trends in WLA / SOAP
The trend towards the cloud continues (e.g. for scalability, resilience), along with container-based architecture for software in the cloud. These changes in the nature and allocation of workloads have led many organisations to rethink their WLA software. In some companies, the desire to move quickly has displaced the desire to standardise on WLA software, as migrations can be complex, disruptive and time-consuming. Instead, these companies acquire new (more) WLA software to address new use cases that require more modern and robust WLA capabilities, but do not immediately plan to migrate the entire legacy production stack to the new WLA software. This leads – as a consequence – to a multi-vendor strategy, not least due to the motive of reducing dependencies and gaining a more robust negotiating position. This is also accompanied by the increasing relevance of SaaS offers.
The trend towards self-servicing or decentralisation continues and is gaining momentum. The “traditional” WLA usage model was that there was a circle of “work schedulers” who dealt with scheduling; these were usually teams of 5 to 15 people. Today, by contrast, WLA implementations in some organisations have hundreds of users from a broader group of IT operations staff, developers, service desk staff and business process owners.
EMA and other market analysts are beginning to describe this software category as an automation orchestrator, in other words: a control layer for automation tools within a company. EMA currently refers to this change as automation orchestration. For example, automation solutions around RPA would also be orchestrated. Basically, this development means that these WLA / SOAP tools must have the ability for comprehensive integration, i.e. a broad set of “out of the box” integrations.AutomateNOW! – a state-of-the-art WLA / SOAP Software
Beta Systems (where I have been Director M&A since 2017), an established provider of data centre software for 40 years, recently acquired the technology leader in WLA / SOAP software. See the following press release: Germany-based Beta Systems acquires InfiniteDATA, aiming for a leading position in the global automation market with AutomateNOW!
Go to the Product website: www.infinitedata.com
Due to its modern cloud-native architecture, the AutomateNOW! product is particularly well positioned to meet the requirements of modern data centre operations:
More to read
