The Umbrella User Guide Developer Hub

Welcome to the Umbrella User Guide developer hub. You'll find comprehensive guides and documentation to help you start working with Umbrella User Guide as quickly as possible, as well as support if you get stuck. Let's jump right in!

Get Started    

Log Format and Versioning

Zipped CSV log files are available for download from either Cisco's managed S3 bucket or your own S3 bucket. Unzipping and opening these files displays multiple columns of information extracted from your Umbrella logs. There are additional fields that are exposed in these logs that are not normally shown through Umbrella's reports. For more information on reporting, see Get Started with Reports.

File Name Format

Logs are uploaded in ten-minute intervals from the Umbrella log queue to the S3 bucket. Within the first two hours after a completed configuration, you should receive your first log upload to your S3 bucket. To check to see if everything is working, the Last Sync time in the Umbrella dashboard should update and logs should begin to appear in your S3 bucket (Amazon S3 > <bucketname> > dnslogs). The logs will appear in a GZIP format with the following file name format. The files will also be sorted into date-stamped folders.

DNS traffic
dnslogs/<year>-<month>-<day>/<year>-<month>-<day>-<hour>-<minute>.csv.gz

Proxied traffic (the intelligent proxy)
proxylogs/<year>-<month>-<day>/<year>-<month>-<day>-<hour>-<minute>.csv.gz

IP traffic generated from the IP Layer enforcement feature (a sub-feature of the intelligent proxy)
iplogs/<year>-<month>-<day>/<year>-<month>-<day>-<hour>-<minute>.csv.gz

Subfolders

Logs are uploaded to S3 buckets in the appropriate subfolder with the following naming format.
<subfolder>/<YYYY>-<MM>-<DD>/<YYYY>-<MM>-<DD>-<hh>-<mm>-<xxxx>.csv.gz

<subfolder> will either be dnslogs, proxylogs, iplogs, and auditlogs depending on the types of logs within. <xxxx> is a random string of four alphanumeric characters, which prevents duplicate file names from being overwritten.
Example: dnslogs/2019-01-01/2019-01-01-00-00-e4e1.csv.gz

Versioning

Depending on the Umbrella subscription you have, and depending on the type of bucket you configure, there are different versions of the log formats. Currently, there are five versions:

  • Version 1—for customers who have configured their own S3 bucket before November 2017. This version has a single sub-folder in the bucket and contains only DNS traffic logs.
  • Version 2—for customers who have configured their own S3 bucket after November 2017, or are using a Cisco-managed bucket. This version is inclusive of everything in version 1, and adds two new log types: Proxy traffic logs and IP traffic logs. Each log type has its own sub-folder.
  • Version 3— the same as version 2, but adds two new fields: Most Granular Identity Type and Identity Types for DNS logs.
  • Version 4—the same as version 3, but adds the Blocked Categories field for DNS and Proxy logs.
  • Version 5—the same as version 4, but adds three new fields: all Identities, all Identity Types, and Request Method for Proxy logs.
  • Version 6—the same as version 5 with these additional fields to Proxy logs: Certificate Errors, Destination Lists IDs, DLP Status, File Name, Rule ID, and Ruleset ID.

Version 1 Bucket Recreation

If you are on version 1, you will need to remove your existing S3 bucket, disable the integration, then create a new bucket from scratch. For all other versions, you can upgrade from the Log Management screen of the Umbrella dashboard by clicking the Upgrade button.

DNS Logs

DNS logs show traffic that has reached our DNS resolvers.
Example:
"2015-01-16 17:48:41","ActiveDirectoryUserName", "ActiveDirectoryUserName,ADSite,Network", "10.10.1.100","24.123.132.133","Allowed","1 (A)", "NOERROR","domain-visited.com.", "Chat,Photo Sharing,Social Networking,Allow List"

  • Action—Whether the request was allowed or blocked.
  • Blocked Categories—The categories that resulted in the destination being blocked. Available in version 4 and above.
  • Categories—The security or content categories that the destination matches. For category definitions, see Understanding Security Categories and Understanding Content Categories.
  • Domain—The domain that was requested.
  • External IP—The external IP address that made the request.
  • Identities—All identities associated with this request.
  • Identity Types—The type of identity that made the request. For example, Roaming Computer, Network, and so on. Available in version 3 and above.
  • Internal IP—The internal IP address that made the request.
  • Most Granular Identity—The first identity matched with this request in order of granularity.
  • Most Granular Identity Type—The first identity type matched with this request in order of granularity. Available in version 3 and above.
  • Query Type—The type of DNS request that was made. For more information, see Common DNS Request Types.
  • Response Code—The DNS return code for this request. For more information, see Common DNS return codes for any DNS service (and Umbrella).
  • Timestamp—When this request was made in UTC. This is different than the Umbrella dashboard, which converts the time to your specified time zone.

Proxy Logs

Proxy logs show traffic that has passed through the Umbrella secure web gateway (SWG) or the Selective Proxy.
Example:
"2017-10-02 23:52:53","TheComputerName","ActiveDirectoryUserName, ADSite,Network","192.192.192.135","1.1.1.91","","ALLOWED", "http://google.com/the.js","www.google.com", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36","200", "562","1489","","","","","","","","Networks"

Not all fields listed here are used in most or all requests and are included for future enhancement.

  • AMP Disposition—The status of the files proxied and scanned by Cisco Advanced Malware Protection (AMP) as part of the Umbrella File Inspection feature; can be Clean, Malicious or Unknown.
  • AMP Malware Name—If Malicious, the name of the malware according to AMP.
  • AMP Score—The score of the malware from AMP. This field is not currently used and will be blank.
  • AV Detections—The detection name according to the antivirus engine used in file inspection.
  • Blocked Categories—The category that resulted in the destination being blocked. Available in version 4 and above.
  • Categories—The security categories for this request, such as Malware.
  • Certificate Errors—Any certificate or protocol errors in the request.
  • Content Type—The type of web content, typically text/html.
  • Destination IP—The destination IP address of the request.
  • Destination List IDs—The ID number umbrella assigns to a destination list.
  • DLP Status—If the request was Blocked for DLP.
  • External IP—The egress IP address of the network where the request originated.
  • File Name—The name of the file.
  • Identities—All identities associated with this request.
  • Identity Types—The type of identities that were associated with the request. For example, Roaming Computer, Network, and so on. Available in version 5 and above.
  • Internal IP—The internal IP address of the computer making the request.
  • Policy Identity—The identity that made the request.
  • Policy Identity Type—The first identity type that made the request. For example, Roaming Computer, Network, and so on.
  • PUAs—A list of all potentially unwanted application (PUA) results for the proxied file as returned by the antivirus scanner.
  • Referer—The referring domain or URL.
  • Request Method—The request method (GET, POST, HEAD, etc.)
  • Request Size (bytes)—Request size in bytes.
  • Response Size (bytes)—Response size in bytes.
  • Response Body Size (bytes)—Response body size in bytes.
  • Rule ID—The ID number assigned to the rule by Umbrella.
  • Ruleset ID—The ID number assigned to the ruleset by Umbrella.
  • SHA—SHA256—The hex digest of the response content.
  • Timestamp—The timestamp of the request transaction in UTC (2015-01-16 17:48:41).
  • Status Code—The HTTP status code; should always be 200 or 201.
  • URL—The URL requested.
  • User Agent—The browser agent that made the request.
  • Verdict—Whether the destination was blocked or allowed.

IP Logs

IP logs show traffic that has been handled by the IP Layer Enforcement feature.
Example:
"2017-10-02 19:58:12","TheComputerName","198.198.198.1", "55605","107.152.24.219","443","Unauthorized IP Tunnel Access"

  • Categories—Which security categories, if any, matched against the destination IP address/port requested.
  • Destination IP—The destination IP requested.
  • Destination Port—The destination port the request was made on.
  • Identity—The first identity matched with this request in order of granularity.
  • Source IP—The IP of the computer making the request.
  • Source Port—The port the request was made on.
  • Timestamp—When this request was made in UTC.

For more information about IP Layer Enforcement, see Add IP Layer Enforcement—DNS Policies Only.

Cloud Firewall Logs

Cloud Firewall logs show traffic that has been handled by network tunnels.
Example:
"2019-01-14 18:03:46","[211039844]","Passive Monitor", "CDFW Tunnel Device","OUTBOUND","1","84","172.17.3.4","","146.112.255.129", "","ams1.edc","12","ALLOW"

  • Data Center—The name of the Umbrella data center that processed the user-generated traffic.
  • Destination IP—The destination IP address of the user-generated traffic towards the CDFW.
  • Destination Port—The destination port number of the user-generated traffic towards the CDFW.
  • Direction—The direction of the packet. It is destined either towards the internet or to the customer's network.
  • Identity—The name of the network tunnel.
  • Identity Type—The type of identity that made the request. Should always be "CDFW Tunnel Device".
  • IP Protocol—The actual protocol of the traffic. It could be TCP, UDP, ICMP.
  • Origin ID—The unique identity of the network tunnel.
  • Packet Size—The size of the packet that Umbrella CDFW received.
  • Rule ID—The ID of the rule that processed the user traffic.
  • Source IP—The internal IP address of the user-generated traffic towards the CDFW. If the traffic goes through NAT before it comes to CDFW, it will be the NAT IP address.
  • Source Port—The internal port number of the user-generated traffic towards the CDFW.
  • Timestamp—The timestamp of the request transaction in UTC.
  • Verdict—The final verdict whether to allow or block the traffic based on the rule.

Admin Audit Logs

Admin Audit logs show changes made by your administrative team in your organization's Umbrella settings.
Example:
"2021-07-22 10:46:45","[email protected]"," ","logexportconfigurations","update","209.165.200.227","version: 4 ","version: 5"

  • Action—The type of change made, such as Create, update, or Delete.
  • After—The policy or setting after the change was made.
  • Before—The policy or setting before the change was made.
  • Logged in from—The user's IP source.
  • Timestamp—The date and time when this request was made in UTC. This is different than the Umbrella dashboard, which converts the time to your specified time zone.
  • Type—Where the change was made, such as settings or a policy.
  • User—The account name of the user who created the change.
  • User Role—The type of role the user has in Umbrella.

Delete Logs < Log Format and Versioning > Configure SAML Integrations

Updated 3 days ago

Log Format and Versioning


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.