Skip to main content

Collecting Telemetry for Air Gap Instances

This topic describes how to collect telemetry for instances in air gap environments.

Overview​

Air gap instances run in environments without outbound internet access. This limitation prevents these instances from periodically sending telemetry to the Replicated Vendor Portal through the Replicated SDK or Replicated KOTS. For more information about how the Vendor Portal collects telemetry from online (internet-connected) instances, see About Instance and Event Data.

For air gap instances, Replicated KOTS and the Replicated SDK collect and store instance telemetry in a Kubernetes Secret in the customer environment. The Replicated SDK also stores any custom metrics within its Secret.

The telemetry and custom metrics stored in the Secret are collected when a support bundle is generated in the environment. When the support bundle is uploaded to the Vendor Portal, the telemetry and custom metrics are associated with the correct customer and instance ID, and the Vendor Portal updates the instance insights and event data accordingly.

The following diagram demonstrates how air gap telemetry is collected and stored by the Replicated SDK in a customer environment, and then shared to the Vendor Portal in a support bundle:

Air gap telemetry collected by the SDK in a support bundle

View a larger version of this image

All support bundles uploaded to the Vendor Portal from air gap customers contributes to a comprehensive dataset, providing parity in the telemetry for air gap and online instances. Replicated recommends that you collect support bundles from air gap customers regularly (monthly or quarterly) to improve the completeness of the dataset. The Vendor Portal handles any overlapping event archives idempotently, ensuring data integrity.

Requirement​

Air gap telemetry has the following requirements:

  • To collect telemetry from air gap instances, one of the following must be installed in the cluster where the instance is running:

    note

    When both the Replicated SDK and KOTS v1.92.1 or later are installed in the cluster (such as when a Helm chart that includes the SDK is installed by KOTS), both collect and store instance telemetry in their own dedicated secret, subject to the size limitation noted below. In the case of any overlapping data points, the Vendor Portal will report these data points chronologically based on their timestamp.

  • To collect custom metrics from air gap instances, the Replicated SDK must installed in the cluster in air gap mode. See Installing the SDK in Air Gap Environments.

    For more information about custom metrics, see Configuring Custom Metrics.

Replicated strongly recommends that all applications include the Replicated SDK because it enables access to both standard instance telemetry and custom metrics for air gap instances.

Limitation​

Telemetry data is capped at 4,000 events or 1MB per Secret; whichever limit is reached first.

When a limit is reached, the oldest events are purged until the payload is within the limit. For optimal use, consider collecting support bundles regularly (monthly or quarterly) from air gap customers.

Collect and View Air Gap Telemetry​

To collect telemetry from air gap instances:

  1. Ask your customer to collect a support bundle using one of the following methods:

    • Run the following command with the support-bundle CLI:

      kubectl support-bundle --load-cluster-specs

      Where --load-cluster-specs is a required flag that ensures telemetry and custom metrics are included in the support bundle that is generated.

      For more information, see Generating Support Bundles.

    • (KOTS Only) For KOTS installations, users can also generate and download a support bundle from the Admin Console Troubleshoot tab to ensure that telemetry and custom metrics are included. See Generating a Support Bundle Using the Admin Console.

  2. After receiving the support bundle from your customer, go to the Vendor Portal Customers, Customer Reporting, or Instance Details page and upload the support bundle:

    upload new bundle button on instance details page

    The telemetry collected from the support bundle appears in the instance data shortly. Allow a few minutes for all data to be processed.