When running a node scan, a number of custom scan options can be specified to give extra visibility into items of particular interest. Scan options are always applied at the node group level to ensure valid comparisons between nodes.

Setting Scan Options

From the Node Page

To view and edit the scan options applied to a node, click the drop down next to the edit button on the node details page.

w300

It will pop up the scan options interface. Here you can see the groups to which the node belongs and what scan options it inherits from each. This will ensure that Nodes will be scanned consistently so that whenever you want to compare two Nodes you can be confident that the important configuration items will be available. You can add or edit scan options here and they will be saved and applied to the parent group the next time those nodes are scanned.

node-group-scan-options-03

From the Node Group Page

You can also view and edit a node group’s scan options from the Node Group list on the side bar of the Monitored Nodes page. Locate the node group, then click the gear icon to select Edit.

w300

Files and Directories

This section allows you to specify which files, or groups of files, are collected up under the Files section of a node scan. Multiple files, or patterns representing groups of files, can be specified one per line.

node-group-scan-options-02

node-group-scan-options-02

Absolute Syntax

To specify a specific file, simply include the absolute path to that file. For example, to scan the UpGuard Agent binary you could include this line:

C:\Program Files\UpGuard\bin\upguard.exe

To request all files within a folder to be scanned, simply include the absolute path to the folder. For example, to include all files in the “C:\Windows\System32” folder use:

C:\Windows\System32

Wildcard Syntax

To request a particular type of file in a given folder you can use the wildcard * character. For example, to include all .exe files in C:\Windows\System32 folder use the following line:

C:\Windows\System32\*.exe

Greedy-star Syntax

To request a particular type of file in a given folder and its subfolders, use the greedy ** wildcard pattern. For example, to include all .exe files in C:\Windows\System32 and all subdirectories use the following line:

C:\Windows\System32\**\*.exe

To include all files, regardless of file extension in C:\Windows\System32 and all of its subdirectories, use the following line:

C:\Windows\System32\**\*

File Exclusion via Path Negation

You can prepend a scan directory option with ! in order to cause files that match the pattern to not be returned as a part of the scan. All forms of globbing for the exclude option will still be supported with file path negation.

If you were to use !C:\Folder\**\*.log, then C:\Folder\Folder2\sample.log will be excluded from the scan, as would C:\Folder\something-else.log, as the greedy-star would help match .log files in that directory and all subdirectories.

As another example, to include all files within a git repository, but exclude the .git/ directory and all contents of that directory you could set up the following scan option path rules:

/home/user/git/project/**/* scan all files in my project directory, including subdirectories
!/home/user/git/project/.git/**/* but, exclude all files and directories inside the project’s .git directory
!/home/user/git/project/.git and exclude the .git directory itself

Order of Precedence of Conflicting Rules

When specifying a number of scan options, particular files can be matched by more than one pattern or path. If your file scan option list does not contain any exclusion rules, then a file matched multiple times will still only be collected once. In essence, the union of all scan rules is returned during a scan.

Things get complicated when exclusion rules are defined. Given the git folder example from above (see File Exclusion via Path Negation), you will notice that the first rule matches all files in a git repository. The subsequent rules then disallow some of those matched files from being included, so the exclusion rules take precendence here.

The order of precendence is as follows:

  • Any path with no stars beats any path with one or more stars,
  • Any path with a single star beats any path with a double star,
  • The longer (more specific) path beats a less specific path, and
  • Lexicographic order.

These rules generally cater for most use cases where a larger, more general pattern is specified, followed by more specific exclusion rules for subsets of matched files.

Custom Order of Precendence

If you have a particular use case that does not fit with the default order of precendence outlined above then you can give specific scan path rules a priority. By default, if a scan option does not have a priority value set, then it defaults to the value 1000. Lower numbers have higher priority.

To set a custom priority on a rule, navigate to the scan option rule and click the pencil edit button. A priority field should allow you to specify an integer value. Here we are setting a particular scan option rule to have a priority value of 40.

w600

Troubleshooting

No files were returned for my scan options even though I see them on the node.

  • Try to list the scan option pattern on the command line of the target node. If on Linux, use the ls command, otherwise on Windows use the dir /b command.
  • Confirm that the user that the scan is run via on the node can actually access the files or directory in question.
  • Add a custom command to ls -l (Linux/Unix) or dir /b (Windows) for the path you want to scan to confirm the Connection Manager has access to the files.

Etcd Keys

This section allows you to specify which etcd key, or groups of etcd keys, will be collected under the “Etcd Keys” section of the node scan. Multiple keys, or patterns representing groups of keys, can be specified one per line.

node-group-scan-options-02

NMAP Scanning

This section allows you to perform external port scanning on Non-Windows based nodes. NMAP scanning is found under the Categories section of Scan Options. Ports for this scan can be designated in the following ways:

  • A single port (5985)
  • A set of ports (5985, 5986, 1433)
  • A range of ports (1-1024)

After specifying the ports needed for NMAP scanning, click the check box to save your preferences.

w400

Custom Scripts (Linux Only)

This section allows you to specify custom scripts that can be used to return data that you would like to see included in the scan. The description field is used to distinguish your query. Queries are interpreted to be bash scripts by default, but it is recommended to specify the shell explicitly at the top of your script (for example #!/bin/bash, #!/usr/bin/env ruby, #!/usr/bin/env python).

Returned scripts are displayed as flat files under a Scripts section of the visualization. This allows you to return back data that may not have a strictly defined structure.

custom-scripts-01

If you would like to display configuration as seperate configuration item squares, refer to the scan.d options method.

SQL Queries (Database Nodes Only)

Adding SQL queries to your scan options allows you to easily detect changes to database table schemas, triggers, stored procedures or indexes. For example, to detect column or attribute changes you can select on your databases schema table. Here we set the schema first for our Microsoft SQL Server database and select our sales column data:

use sales select * from information_schema.columns

PowerShell Queries (Windows Only)

Custom PowerShell queries can be used to return data you would like to see included in the scan.

Field Description
Description The label given to the results in the scan.
Key Name If the query returns multiple values, Key Name is used to specify which field uniquely identifies the row.
Query The actual PowerShell query to be run.

Single Object Example

Field Description
Description Host
Key Name  
Query Get-Host | Select CurrentCulture, Name, Version

Multiple Objects Example

Note the Try/Catch used due to a PowerShell bug.

Field Description
Description Websites
Key Name PhysicalPath
Query Import-Module WebAdministration; Try{ $sites = Get-WebSite | Select Name, State, PhysicalPath, ApplicationPool } Catch [System.IO.FileNotFoundException]{ $site = Get-WebSite | Select Name, State, PhysicalPath, ApplicationPool } $sites

Registry Keys (Windows Only)

Specify registry keys to be scanned. Valid options are full paths to a key value name or parent key path. Abbreviations such as HKLM are supported.

Key Value

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\Tcpip\Parameters\DataBasePath

Parent Key Value

HKLM\SYSTEM\CurrentControlSet\services\Tcpip\Parameters

Scanning Multiple Keys

UpGuard’s registry scanning supports glob syntax for scanning multiple registry sub keys. For sub keys, utilizing * will function as an wildcard, and utilizing ** will function as a recursive wildcard.

w300

Group Policy Objects (Windows Only)

Group Policy Objects (GPOs) can be scanned by entering the name of the GPO as it appears in the Windows Group Policy Management Console (GPMC).

scan-options-gpo node-group-scan-options-04

Instructions for getting started with the GPMC can be found here.

Resultant Set of Policy (RSoP) (Windows Only)

Resultant Set of Policy (RSoP) reports can be used to identify the GPO’s that are in actuality governing a machine.

RSoP reports can be accessed by adding “RSOP” as a scan option under the “Sections” category.

node-group-scan-options-07

The results of the RSoP report are visualized as a configuration group in the node visualization.

By treating RSoP results as configuration items in UpGuard, users can generate policies, compare differences, identify governance inconsistencies, and integrate information with additional platforms.

node-group-scan-options-08

Troubleshooting

I receive a warning message after a scan looking like “Cannot find path ‘C:\Windows\Temp...\RSOPReport’ because it does not exist.”

  • This message is printed when the user the UpGuard service is being run under does not have access to enough RSoP data to generate a valid report.
  • Windows Agent version 4.21.0 onwards replaces this error message with an empty RSoP section in the resulting scan and a warning message indicating that the particular user doesn’t have access to RSoP data.

I can’t see an RSoP section in the scan or the section is empty.

  • Confirm that the scan section “RSOP” can be added for the node being scanned.
  • Check the connection manager or agent logs after a scan, at C:\Program Files\UpGuard\logs. Check for a line looking like Warn: The user ... does not have RSoP data. This implies the user account the agent or service is running under does not have access to any valid RSoP data to generate a valid report. You can alter the user the UpGuard Service runs under by right clicking the UpGuard service in the Services window, selecting properties and altering the user the service is run as.

IIS Settings (Windows Only)

Adding IIS settings can be done through Scan Options. While logged in as an Administrator, go to the Discover tab, and choose Monitored. For the node or node group you would like to monitor IIS settings on, click on the dropdown menu next to the Edit button near the top right hand corner and select Scan Options.

Navigate to the Sections tab and enter “IIS” (case sensitive) as a section and click the check box to save the newly added option.

node-group-scan-options-08

Exit out of Scan Options and run a scan on the node or node group. IIS settings will be listed after a successful scan.

node-group-scan-options-08

Ports

By default, active local TCP and UDP ports in the range 1-1024 (inclusive) are scanned. To specify a particular range of ports, simply use the M-N syntax. For example, to scan all ports between 20 and 30 (inclusive), use 20-30. You can also specify a number of individual ports, or ranges of ports, by separating them by a comma. For example, to scan ports 22, 80, 443 and all ports in the 8000s use 22,80,443,8000-8999.

Connectivity

The connectivity tests allow you to determine if a node is able to establish a remote connection on the specified port. The host may be specified using either IP address or FQDN.

node-group-scan-options-05

Web

The Web scan option allows you to verify that a node can connect to a web endpoint, and optionally retrieve the contents of the response body. The label field is used to identify the check within the scan output.

node-group-scan-options-06

Excluded Text

The “Excluded Text” scan option allows users define a set of textual patterns that will be excluded from differencing operations. These patterns are defined using regular expressions, and are applied to the scan data at comparison time, so no data is lost. Both attributes and raw files will be considered for text exclusion operations; regardless of the place of application, if applying exclusion rules results in no difference, no difference will be surfaced visually or as part of the event system.

excluded-text

Scan Option Variables

The following variables can be used in the scan options page to help parameterize queries. Variables can be used at the node and node group levels and must be used using liquid syntax.

  • {{ node_name }}
  • {{ node_environment }}
  • {{ hostname }}
  • {{ ipaddess }}
  • {{ operatingsystem }}
  • {{ osfamily }}