Troubleshoot Resource Connectors and Connector Groups

Use the following information to troubleshoot connector issues:

• General Troubleshooting
• Throughput Capacity is Less Than Expected
• Users Cannot Connect to Private Resources
• Connector software Auto-Upgrade Fails
• Connector Operating System (OS) Version has Security Vulnerabilities
• Connector is Expired
• Stop a Connector
• Unable to Revoke or Delete a Connector
• Connector-Related Status Graphs are not Current
• A Note about Connector Issues
• Connector Diagnostics (CLI)

General Troubleshooting

If you are seeing issues with connectors or connector groups, try the following:

• Check the Connectivity section on the Overview page.
  This provides a high-level view of the status of your organization's configured connectors and connector groups.
• Check the Connector Groups page for issues:
  Navigate to Connect > Network Connections > Connector Groups and look at the Status column for the applicable connector group.
• Check Monitor > Activity Search for logged events related to connectors.
• Determine whether the issue affects a single connector or all connectors in the group.
  If the issue affects only one connector, it may be simplest to delete the connector and deploy a new one.
• Check for a Secure Access service outage. See View Cloud Security Service Status.
• Temporarily disable a connector while you test the situation, then re-enable it as appropriate.
• Run Connector Diagnostics (CLI).

Note: Some troubleshooting guidance outside of this document may refer to connectors as "agents", "app connectors", or "application connectors".

Throughput Capacity is Less Than Expected

Ensure that connectors are using DTLS and not TLS:

Run Connector Diagnostics (CLI) on the connector and look for Protocol in the output.

If the protocol shows TLS, check the following:

• DTLS and TLS connections must originate from the same IP address.
• The egress firewall must allow UDP connections on port 443.
• The connector is the required instance type.
  See Requirements and Prerequisites for Resource Connectors and Connector Groups.
• In the connector instance, "Auto-Assign Public IP" must be enabled.

If you make changes, reboot the instance.

Users Cannot Connect to Private Resources

• Ensure that you have assigned the resource to the applicable connector group. See Assign Private Resources to a Connector Group.
• Verify that all connectors in the group can reach the resource.
• Check the status of the connector group that the resource is associated with:
  Navigate to Connect > Network Connections > Connector Groups and look at the Status column for the applicable connector group.
• For causes of user connectivity issues that are not specifically related to resource connectors, see Troubleshoot Private Access Rules.

Connector Software Auto-Upgrade Fails

For general information about automatic connector updates, see Maintain Resource Connectors.

• The upgrade problem may be temporary and self-repairing. Secure Access automatically periodically retries the upgrade.
• If upgrade fails for a single connector, this will prevent upgrades for all connectors in the group that have not yet been upgraded.
• To determine the problem connector, look for a yellow icon in the Version column on the connector group page:.
• Try deleting the problem connector instance and deploying a new one. See Disable, Revoke, or Delete Resource Connectors and Groups.
• Make sure the connector can connect to the destinations in Allow Resource Connector Traffic to Secure Access.
• If the upgrade continues to fail:
  • Check for problems with your network or the connector instances themselves (such as a full disk.)
  • Run Connector Diagnostics (CLI).

Connector Operating System (OS) Version has Security Vulnerabilities

Connector version issues are indicated in a list of connectors with a yellow icon ( ) Hover over the icon to identify the problem.

If you see a yellow icon for a connector version on the connector group's list of connectors, the platform operating system may have vulnerabilies. To determine whether this is the case, hover over the status icon. If OS vulnerabilities have been reported, you should replace affected connectors.

To replace connectors, see the "Connector platform operating system (OS) updates" section in Maintain and Monitor Resource Connectors and Connector Groups.

Connector is Expired

To ensure the integrity of your deployed connectors, Secure Access validates each connector during the connector's renewal period. The renewal period spans several weeks where Secure Access checks the status of a deployed connector. Toward the end of the renewal period if the connector's state is not valid, Secure Access disconnects the resource connector and the connector expires.

Check the Status of Your Connector

We recommend that you log into a deployed resource connector periodically and check the status of the connector.

1. Log in to the virtual machine for the resource connector.
2. In the connector's diagnostic output, locate sse_cloud, and then review the resource connector's diagnostic information.

If your resource connectors expire frequently, we recommend that you review the prerequisites for deploying resource connectors in your environment. For more information, see Requirements and Prerequisites for Resource Connectors and Connector Groups.

(VMware Only) View Connector Diagnostic Information

The resource connector console on VMware displays diagnostic information before the resource connector is shutdown.

For example:

systemd-shutdown[1]\: Failed to acquire terminal, using /dev/null stdin/stdout/stderr instead: Input/output error

Check for an Expired Connector in Secure Access

1. Navigate to Connect > Network Connections > Connector Groups.
2. Locate the Connector Group table and click on a Connector Group.
3. Locate the connector and confirm that its Status is Expired.

Clean Up an Expired Connector

1. Delete the expired connector instance from your virtual environment.
2. Delete the expired connector from the connector group in Secure Access. For more information, see Disable, Revoke, or Delete Resource Connectors and Groups.
3. If needed, deploy a new connector. For more information, see Add Connectors to a Connector Group.

Stop a Connector

if you need to stop a connector for any reason, power it off from within the deployment environment (AWS, VMware, etc.)

Unable to Revoke or Delete a Connector

This includes the inability of Secure Access to automatically delete an inactive connector.

This situation is likely to be temporary and self-repairing, and can occur even if the connector is still passing traffic to private resources. If the problem does not resolve itself, run Connector Diagnostics (CLI).

Connector-Related Status Graphs are not Current

For example, the CPU Load chart does not have current data, or the graphs that you see when you click a connector ID in the list of connectors in a group are not up to date.

This situation is likely to be temporary and self-repairing, and can occur even if the connector is still passing traffic to private resources. If the problem does not resolve itself, run Connector Diagnostics (CLI).

A Note About Connector Issues

Connectors may be passing traffic successfully but not communicating otherwise with the Secure Access cloud.

This can cause problems with automatic software updates, monitoring connector health, gathering metrics, and deleting or revoking a connector, including automatically deleting inactive connectors.

These issues are typically temporary and self-repairing.

If they are not, first try restarting the connector instance.

If the issues continue, you can run connector diagnostics to troubleshoot these issues. See Connector Diagnostics (CLI).

Connector Diagnostics (CLI)

To troubleshoot connector issues that cannot be resolved by restarting or redeploying the connector instance, run diagnostic commands on the connector instance.

Supported Commands:

Run the Diagnostic Command

To run the diagnostic command:

1. Run the connector diagnostic tool from the connector's command line:
   1. SSH to the connector instance.
      Use the SSH key you configured while deploying the connector instance.
      Sign in with user name acadmin.
   2. At the $ prompt, enter diagnostic or another command of interest.
      To understand the results of the diagnostic command, see the Diagnostic Codes tables below.
2. If the issue persists, contact Support.

Diagnostic Codes

When you run the diagnostic command as described above, you will see state codes as described in the following tables.

If an issue is not listed in the following tables, or an issue persists after you have taken the recommended action, contact Cisco TAC.

Diagnostic Codes for Connector Update Issues

Diagnostic Codes for Other Connector Issues

Supported Standard Linux Troubleshooting Commands

You can use the following commands on a connector instance:

ping, nslookup, traceroute, tcptraceroute, netstat, free, df, vmstat, iostat, mpstat, reboot, uptime, date, clear

---

Maintain and Monitor Resource Connectors and Connector Group < Troubleshoot Resource Connectors and Connector Groups > Secure Access Regions