Auto-Sync PowerShell Modules of Azure Automation Hybrid Workers from Automation Account.

In my previous article we have seen how we can use Azure Automation State Configuration to automate the deployment and monitoring of Self-hosted agents in Azure DevOps. In this article, we will see how we can automatically sync PowerShell modules of hybrid workers from automation account to using state configuration.

Before proceeding further…

  • You need to have a basic understanding of Azure Automation and Desired State Configuration.
  • You need to have an Azure Subscription. Create one from this link if you do not have any.
  • You need to have an Azure Automation Account and Windows VM which needs to be used as hybrid worker deployed in a resource group.

A Hybrid Worker is a feature offered by Azure Automation, where the users can bring and connect their own compute resource to automation account for job execution. Hybrid workers help in overcoming few resource limitations that are applicable to azure sandboxes. They also come handy in scenarios where the runbook needs to access any on-premise resource which is not possible in azure sandboxes.

Apart from these there are many other scenarios where the hybrid workers are recommended. More details can be found here.

A runbook defined in PowerShell can have dependency on PowerShell modules. If a runbook job has to be executed with out failure then all it dependency modules have to installed in the environment where the job is executed. This process is different for azure sandboxes and hybrid workers.

For azure sandboxes, it is handled by the automation service and all the dependency modules are automatically installed into the sandbox environment if the module is uploaded to automation account. But if the same modules have to be used then the user has to ensure that they are installed on all the worker machines.

This is a very tedious work as a user could have huge number of workers and performing this operation manually on all will be time consuming. For custom modules the process will be even more difficult as the modules have to be manually copied to each worker machine. This also not a one time activity as there are frequent updates to azure modules are ensuring that the workers are in sync with automation account for each update is difficult.

What if we could sync the modules state of hybrid workers with that of the automation account with a single click of a button. We try to achieve the same with the help of automation state configuration.

Let us first try to understand how automation state configuration works before jumping into the solution...

Automation State Configuration is an implementation of DSC (Desired State Configuration) pull server that can be used to onboard nodes to DSC in pull mode.

DSC Pull Server:
A DSC pull server is a remote service that is provisioned with configurations and resources. This service exposes few endpoints which are used by the dsc nodes to download configurations and resources, in order to ensure that the nodes are in desired state. DSC protocol defines contracts for each of the endpoint and every pull server implementation should follow these contracts.

As automation state configuration implements dsc pull server, it has endpoints for each of the dsc protocol. Among these endpoints, there is an endpoint (ModuleContent, a GET request) which is used to download the modules for dsc (otherwise called as dsc resources) from the pull server and install them on the node. The pull server url that is used by the dsc to download the modules is configured in ‘ResourceModuleManagers’ settings of LCM

Since the hybrid worker node is being onboarded to automation state configuration, it is already able to download modules for dsc configurations from automation account. We will extend this logic to download any desired modules and sync the modules state.

Since none of the existing resource can serve our requirement we need to either use script resource or define a new resource. I have chosen the later option so that we can add additional functionality to the module in future. This documentation details the steps for creating a new resource.

New DSC Resource

I have defined a new resource ‘AutomationPSModuleResource’ under module ‘AutomationHybridWorkerDsc’ with below logic.

“If the module is not installed, the ResourceModuleManagers is retrieved from the LCM configuration and then a Get request is made for the required module and version. Once the module is downloaded it is installed in the location C:\Program Files\WindowsPowerShell\Modules”

The resource has following four properties:

  • Name: The name of the module to be installed.
  • RequiredVersion: The version of the module to be installed.
  • Ensure: Present or Absent the state of the module.
  • ModuleBasePath: The location where the module is installed. Readonly property.

Source code is shown below…

DSC Configuration for Module Sync

Now that we have created a new resource to download and install a module from automation account, let us use this resource to define a configuration which includes all the modules that are currently imported to the automation account. Since any automation account has few default modules used by the service itself, we need to ignore them from the configuration. To achieve this we get all the modules that are uploaded to an account by connecting to the azure subscription and loop through the module list to create the config.

Below configuration for syncing all modules. It assumes that the automation account name and its resource group name are created as variables.

Import and Compile the DSC Configuration

Let us now, import the configuration that we have created so far into an azure automation account and compile it.

  1. Create Automation Variables to store account name and account resource group name.
Variables created in automation

2. Import ‘SyncHybridWorkerModules.ps1’ that we have define above.

# Import the DSC configuration
Import-AzAutomationDscConfiguration `
-ResourceGroupName <AutomationResourceGroup> `
–AutomationAccountName <AutomationName> `
-SourcePath <Path-To-SyncHybridWorkerModules.ps1> `
-Force -Published

3. Compile the DSC configuration. If there is a need to skip any modules from being synced, they need to be passed in to the configuration as parameters. Here we do not pass any parameter so all modules will be synced.

# Compile the DSC configuration
Start-AzAutomationDscCompilationJob `
-ResourceGroupName <AutomationResourceGroup> `
-AutomationAccountName <AutomationName> `
-ConfigurationName 'SyncHybridWorkerModules'

Note:
- The configuration requires AutomationHybridWorkerDsc, Az.Automation and Az.Account modules to be imported. Ensure they are imported before you compile the configuration.
- The configuration uses managed identity to connect to azure and retrieve the current modules of account. Ensure that identity is enabled and required permissions are provided to it on account resource. You can learn more
here.

System Identity enabled in automation

Once the configuration compilation is completed, there should be able to see the node configuration.

Compiled DSC Node Configurations in automation

Register VM as Automation Hybrid Worker

We need to register a VM as a hybrid worker to the automation account. I am skipping the steps as a detailed documentation is available here.

Hybrid Worker registered in automation

Let us say, we have a runbook ‘Audit-AzureResources.ps1’ which queries all the azure resources and reports resources which do not have a owner tag created. For reporting, it uses a custom module ‘Audit.Resources’. This is a very simple module with logs the resource state to console.

We will import all the dependency modules (Az.Accounts, Az.Resources and Audit.Resources) to the account and start the runbook in the azure. We can see that it is successful.

Required modules imported into automation
Runbook executed successfully on azure

If we try to run the same runbook on hybrid worker we can see that it will fail as the modules are not installed on the worker.

Runbook failed on hybrid worker
List of modules installed on hybrid worker before DSC

Onboard the Hybrid Worker to Automation DSC

We have node configuration needed for module sync to hybrid worker. Let us now onboard the hybrid worker to Automation DSC. We are using ‘ApplyAndAutoCorrect’ as the configuration mode, this will apply the configuration and also ensure that the modules are always installed even if someone manually deletes them.

Register-AzAutomationDscNode `
-ResourceGroupName <AutomationResourceGroup> `
-AutomationAccountName <AutomationName> `
-AzureVMName <AzureVMName> `
-AzureVMResourceGroup <AzureVMResourceGroup> `
-AzureVMLocation <AzureVMLocation> `
-NodeConfigurationName 'SyncHybridWorkerModules.localhost' `
-ConfigurationMode 'ApplyAndAutoCorrect' `
-ConfigurationModeFrequencyMins 15 `
-RefreshFrequencyMins 30

Once the node is registered and reported compliant to automation dsc, you should be able to see that all the modules imported in automation are synced to the hybrid worker.

Hybrid Worker onboarded to DSC and reporting compliant
DSC node report showing the modules that are synced
List of modules installed on hybrid worker after DSC

If we try to run our runbook in the hybrid worker it should be successful with out any errors.

Runbook executed successfully on hybrid worker

Update or Add Modules After Initial Sync

For adding new modules or updating existing modules simply do the operation in automation account and the re-compile the configuration. The node status will change to Pending and DSC will automatically update the modules as needed on the hybrid workers and report compliant.

DSC status showing pending after modules are updated

Source Code

Software Engineer II at Microsoft Compute. I strongly believe in quote “Improvise, Adapt and Overcome”. Improvise Skills, Adapt to Changes and Overcome Fears.