Log Formats and Versioning
Secure Access stores zipped CSV log files in Cisco's managed AWS S3 bucket or your own AWS S3 bucket. Each log file displays multiple columns of information that are extracted from your Secure Access logs. A log file may have additional fields that are not shown in the Secure Access reports.
The latest version of the Secure Access log format is v10.
Note: Logs are not always chronological and are not always in the specific time bucket based on the timestamp of the log event.
Table of Contents
- Prerequisites
- Log File Name Formats
- Find Your Log Schema Version
- Log File Fields
- Estimate the Size of a Log
- Estimate the Size of an Exported Report
Prerequisites
- Full Admin user role. For more information, see Manage Accounts.
- Configure logging to an Amazon S3 bucket. See Enable Logging to Your Own S3 Bucket or Enable Logging to a Cisco-managed S3 Bucket.
Log File Name Formats
Logs are uploaded in 10-minute intervals from the Secure Access log queue to the AWS S3 bucket. Within the first two hours after a completed configuration, you should receive your first log upload to your AWS S3 bucket.
Note: To check if everything is working, the Last Sync time in Secure Access should update and logs should begin to appear in your AWS S3 bucket. The logs appear in a GZIP format with the following file name format. The files will also be sorted into date-stamped folders.
- Admin Audit logs
auditlogs/<year>-<month>-<day>/<year>-<month>-<day>-<hour>-<minute>.csv.gz
- Cloud Firewall traffic
firewalllogs/<year>-<month>-<day>/<year>-<month>-<day>-<hour>-<minute>.csv.gz
- DLP traffic
dlplogs/<year>-<month>-<day>/<year>-<month>-<day>-<hour>-<minute>.csv.gz
- DNS traffic
dnslogs/<year>-<month>-<day>/<year>-<month>-<day>-<hour>-<minute>.csv.gz
- File Events traffic
fileeventlogs/<year>-<month>-<day>/<year>-<month>-<day>-<hour>-<minute>.csv.gz
- IPS traffic
intrusionlogs/<year>-<month>-<day>/<year>-<month>-<day>-<hour>-<minute>.csv.gz
- Remote Access VPN traffic
ravpnlogs/<year>-<month>-<day>/<year>-<month>-<day>-<hour>-<minute>.csv.gz
- Web traffic
proxylogs/<year>-<month>-<day>/<year>-<month>-<day>-<hour>-<minute>.csv.gz
- Zero Trust Network Access traffic
ztnalogs/<year>-<month>-<day>/<year>-<month>-<day>-<hour>-<minute>.csv.gz
Subfolders
Logs are uploaded to AWS S3 buckets in the appropriate subfolder with the following naming format.
<subfolder>/<YYYY>-<MM>-<DD>/<YYYY>-<MM>-<DD>-<hh>-<mm>-<xxxx>.csv.gz
Secure Access names a log subfolder (<subfolder>
) with one of the following folder names:
dnslogs
proxylogs
firewalllogs
fileeventlogs
intrusionlogs
auditlogs
dlplogs
ravpnlogs
ztnalogs
The segment of the log GZIP file name is a random string of four alphanumeric characters, which prevents duplicate file names from being overwritten.
Example: dnslogs/2019-01-01/2019-01-01-00-00-e4e1.csv.gz
Find Your Log Schema Version
Depending on the Secure Access subscription that you have, and depending on the type of AWS S3 bucket you configure, there are different versions of the log schemas available. Once your system is configured to log to an AWS S3 bucket you can view the log schema version in use.
Log Schema Versions
-
v1—For customers who have configured their own S3 bucket before November 2017.
Note: To upgrade from v1 to a higher version of the Secure Access log format, you must remove your existing S3 bucket, disable the integration, and then recreate a new bucket. For all other versions, you can upgrade from the Log Management screen of the Secure Access dashboard by clicking Upgrade. -
v2—For customers who have configured their own S3 bucket after November 2017, or are using a Cisco-managed bucket. This version is inclusive of everything in version 1.
-
v3— The same as version 2, but adds two new fields: Most Granular Identity Type and Identity Types for DNS logs.
-
v4—The same as version 3, but adds the Blocked Categories field for DNS and Proxy logs.
-
v5—The same as version 4, but adds three new fields: all Identities, all Identity Types, and Request Method for Proxy logs.
-
v6—The same as version 5, but adds the following fields to Proxy logs: Certificate Errors, Destination Lists IDs, DLP Status, File Name, Rule ID, and Ruleset ID.
-
v7—The same as version 6, but adds the DLP file label field.
-
v8—The same as version 7, but adds fields to the Proxy, DLP, and Firewall logs.
- Proxy logs—Adds the Isolate Action, File Action, and Warn Status fields.
- DLP logs—Changes the Event Type field. The event type is either
Real Time
orSaaS API
. - Firewall logs—Adds the FQDNs and Destination List IDs fields.
-
v9—The same fields as version 8, but the v9 log format adds fields to the IPS and Web logs.
- Adds new fields to the Intrusion Prevention System (IPS) logs: operation mode, policy resource ID, direction, firewall rule ID, IPS config type, AWS region.
- Adds new fields to the Web logs: forwarding method and producer.
- Adds File Events logs.
- Adds Remote Access Virtual Private Network logs.
- Adds Zero Trust Network Access logs.
-
v10—The same fields as version 9, but the v10 log format adds the following fields:
- Cloud Firewall logs: app id, private resource id, private app group id, private flow, posture id, casi category ids, traffic source, content category ids, content category list ids, organization id.
- Data Loss Prevention (DLP) logs: application category name, traffic direction, private resource name, private resource group name, destination protocol, destination ip, destination port, organization id.
- DNS logs: rule id, destination countries, organization id.
- IPS logs: application id, casi category ids, data center, organization id.
- Remote Access VPN logs: asa syslog id, device id, machine id, public ipv6, assigned ipv6, security group tag, dap record name, dap connection type, failed reasons.
- Web logs: msp organization id, geo location, blocked destination countries, application ids, hostname, data center, egress, server name, time based rule, security overridden, detected response file type, warn categories, organization id.
- Zero Trust Access logs: transaction id, block reason, application port, application protocol, tunnel type, secure client version, possible match ruleset id, possible match rule id, possible match posture, source process id, source process name, source process hash, source process user name, organization id, ad joined id.
View Your Log Schema Version and Last Sync Time
- Navigate to Admin > Log Management.
- In the Amazon S3 area, Secure Access displays the log Schema Version in use and Last Sync time.
Last Sync is the date and time of the last sync of your logs to Amazon S3.
Log File Fields
Each type of Secure Access log contains various log fields. Not all field values are available in every log record. When a field does not have a value, Secure Access sets the field to the empty string (""
). For information about the formats of the Secure Access reports, see Reports and CSV Formats.
- Admin Audit Log Formats
- Cloud Firewall Log Formats
- DLP Log Formats
- DNS Log Formats
- File Events Log Formats
- IPS Log Formats
- Remote Access VPN Log Formats
- Web Log Formats
- Zero Trust Network Access Log Formats
Estimate the Size of a Log
The size of your S3 logs depends on the number of events that occur and the volume of the traffic in your organization.
- Download one of your Secure Access log files. The Secure Access log file is a comma-separated values (CSV) file.
- Count the number of rows in the CSV file minus one for the header row.
The number of rows is equivalent to the number of events in the twenty-four hour period. - Multiply the number of rows of data by the number of bytes of data listed in a single row in the file.
The result is the estimate of the size of the event log recorded for one day.
Estimate the Size of an Exported Report
The size of an exported report depends on the number of events that occur, which is dependent on the volume of your traffic. The size of each log line varies based on a number of items—for example, the length of the domain name or the number of categories. Assuming each log line is 220 bytes, a million requests would be 220 MB.
- Navigate to Monitor > Activity Search.
- For Filters, run a report for the last 24 hours and then click the Export CSV icon.
- Open the downloaded CSV file. The number of rows (minus one for the header) is the number of queries per day. Multiply the number of rows by the number of bytes in one row to get the estimate for one day.
Delete Logs < Log Format and Versioning > Reports and CSV Formats
Updated about 2 months ago