The Umbrella Deployment Documentation Developer Hub

Welcome to the Umbrella Deployment Documentation developer hub. You'll find comprehensive guides and documentation to help you start working with Umbrella Deployment Documentation as quickly as possible, as well as support if you get stuck. Let's jump right in!

Get Started    

Enable the Blocking of URLs

Custom URL Destination lists allow Umbrella to extend a domain level destination list to encompass full URLs. In turn, this allows you to block or allow certain parts of a website based specifically on the full URL of that portion of the website.

Feature release timing and Umbrella packages

The custom URL destination feature is only available for customers with the Umbrella Insights or Umbrella Platform packages. For more information about packages, see Umbrella Package Comparison. Contact your Cisco account representative with any questions.

This functionality is being slowly rolled out to individual customers over an extended period of time. Our engineering and networking teams are scaling the service out organically and on-boarding customers slowly to this feature to ensure reliable service as we grow. As a result, you may not see the feature in your dashboard even if you have the package available. The feature will become available to you over the upcoming weeks.

Block a URL

To block a URL, simply enter it into a blocked destination list, or create a new blocked destination list just for URLs.

  1. Navigate to Policies > Policy Components > Destination Lists, expand a Destination list, add a URL and then click Save.
    Note: For this to work as expected, you must adhere to the requirements listed in the sections below.

Implementing Destination Lists with URLs to be Blocked

In order for the Umbrella infrastructure to inspect a URL to determine if it matches the ones defined in your blocked destination list, you must have the following enabled:

  • The intelligent proxy and SSL decryption must be enabled for the policy. For more information, see Step 3: Determine What You Want This Policy To Do.
  • The Cisco Umbrella Root CA must be installed on the computer(s) using this policy—ensures HTTPS connections are filtered, too. For more information, see Cisco Certificate Import Information.
  • It’s important to specify a URL correctly so that what’s in your policy matches what the user is trying to access (and is subsequently blocked).

The reason that SSL decryption and the certificate are required is two fold. First, the certificate is required in order to prevent problems when accessing SSL sites through the intelligent proxy and for SSL decryption to work. Second, the custom URL destination list is protocol agnostic. We don't expect that it's possible to know which protocol a website will use in advance and adding a protocol in front of a URL is not required when defining a destination list. Instead, with the SSL decryption enabled, we're able to block a URL whether it's HTTP or HTTPS and minimize the difficulty of creating a destination list on your end.

URL Normalization for Destination Lists

Umbrella URL Filtering conforms to URL Normalization standards. There are certain guidelines that must be followed in order to ensure the URL you are entering is actually what you want blocked or allowed. These can sometimes mean that the way a URL is displayed in the browser's address bar is not how it should be specified in a destination list.

This is not done automatically. You must format the URL following the guidelines below in order for it to be blocked as expected. For guidelines, see URL Normalization.

For a list of errors generated by incorrect URL addition or other reasons, see Understanding Destination list error messages for custom block URLs.

Troubleshooting if a URL is not blocked

If you do not see the block page when navigating to a URL you expect to be blocked, ensure that the policy with destination list enabled is higher in the policy order than other policies the enrolled identity(ies) are configured for.

Wait upwards of five minutes before testing again after any policy changes to ensure enough time has passed for the changes to be replicated throughout the Umbrella infrastructure.

If problems persist, try clearing the local browser cache on your machine, or even your machine's DNS cache (a reboot will accomplish this).

Beyond that, check to see if you have a local (on-premise) proxy that is interfering. For more information, read this.
And be sure to that the Cisco Umbrella Root CA is installed in case of cert errors. For more information, read this.

It's worth checking the reports and ensuring your URL is correctly normalized. The next two sections will cover this.

Reporting for blocked URLs

A URL is something you can filter against in the Activity Search in the upper right-hand corner:

Once you've filtered for URLs, then just add another filter to show the custom block URLs that belong to your destination lists.

URL Normalization

URLs normalize automatically using the following criteria:

URL Normalization

Protocol Schema (the protocol should be stripped)

hxxp://xyz.com/test → xyz.com/test

Username:Password (should be stripped)

user:pass@xyz.com → xyz.com

Ports (should be stripped)

xyz.com:8080/abc → xyz.com/abc

Trailing slashes (stripped from the end of the URL)

xyz.com/abc/ → xyz.com/abc

Character case (normalized to all lower case)

XYZ.cOm/abC → xyz.com/abc

Examples

The examples below show a single URL and provide examples of what you can and cannot enter to have a block of that URL enforced. The list of URL is built-out from a single example, modifying a single parameter to show whether the URL "//a.co/cx/15195/100/setup_1848x19m.exe?z=z&super=bad&test=yes" would be blocked based on the URL entered in the table below.

In all of these examples, the protocol is stripped as it would be by the interface.

If you wanted to block this URL a.co/cx/15195/100/setup_1848x19m.exe?z=z&super=bad&test=yes the following logic applies:

URL
Will be blocked?
Reason

a.co/cx/15195/100/setup_1848x19m.exe?z=z&super=bad&test=yes

Yes

The full URL is entered.

a.co/cx/15195/100/setup_1848x19m.exe?super=bad&test=yes&z=z

Yes

"&" is a delimiter; therefore, it's added as another level to the URL after the word "yes".

a.co/cx/15195/100/setup_1848x19m.exe?super=bad&test=yes

No

"?" is a delimiter so the URL still would begin at the "yes" and any enforcement would happen after that.

a.co/cx/15195/100/setup_1848x19m.exe?

No

Given the"?", it still means only characters after "yes" will be enforced; therefore, a direct download of this file would be allowed.

a.co/cx/15195/100/setup_1848x19m.exe

No

We will still only block any paths after "yes"; therefore, a direct download of this file would be allowed.

If you wanted to block this URL g.com/a/d, the following logic applies.

These are just examples of which destination list entries would block the URL "g.com/a/d" and which would not.

URL
Will be blocked?
Reason

g.com/a/d

Yes

The full URL is entered.

g.com/a/d?g

Yes

Delimits the path with the query "g" but still just a delimiter thus this will be enforced.

g.com/a/d?

Yes

URL + the "?" delimiter.

g.com/a/

No

The URL ends with "/d" so anything before "/d" would not be enforced.

g.com/a/?a

No

The URL ends with "/d" so anything before "/d" would not be enforced.

If you wanted to block this URL d.co/cx/15195/100 , the following applies. These are just examples of which destination list entries would block the URL "d.co/cx/15195/100" and which would not.

URL
Enforced
Reason

d.co/cx/15195/100

Yes

The full URL is entered.

d.co/cx/15195/100/?

Yes

Everything after the delimited "/" after 100 would be blocked.

d.co/cx/15195/100/

Yes

Everything after the delimited "/" after 100 would be blocked.

d.co/cx/15195/100

Yes

Everything after the delimiting "/" after 100 would be blocked.

d.co/cx/15195/10

No

The delimiter is only for paths after the "/" so any changes to the final path of /100/ would be ignored.

d.co/cx/15195/1000

No

The delimiter is only for paths after the "/" so any changes to the final path of /100/ would be ignored.

d.co/cx/15195/

No

The delimiter is only for paths after the "/" so any changes to the final path of /100/ would be ignored.

d.co/cx/15195

No

The delimiter is only for paths after the "/" so any changes to the final path of /100/ would be ignored.

Further Information

There are some other normalization rules that we don't expect most customers to come across. If you find that a URL you entered is not being properly filtered on and you made sure all the above criteria are met, you may want to look at the URL Normalization RFC for more information.

Getting Help

If you encounter any issues with the custom URL feature, please log a case with Customer Support at: umbrella-support@cisco.com

You may want to include the output of the following commands (these commands should be run from a device enrolled in the policy configured for custom URL blocking):

OS X:
dig proxy.opendnstest.com
dig debug.opendns.com txt

Windows
nslookup proxy.opendnstest.com
nslookup -type=txt debug.opendns.com

You can also include the output of the Umbrella diagnostic to speed up the troubleshooting.


Download Destinations to a CSV File < Enable the Blocking of URLs > Wildcards and Destination Lists