Common Issues and Solutions (Beta)

Beta

The Historical User Activity feature is currently in beta for the Analytics tool.


Review the following table for troubleshooting steps and resolutions for common errors or unexpected behaviors you may encounter during or after upgrading to the latest version of the Analytics Cloud Connector CLI.

Issue

Cause

Solution

"Too many open files (os error 24)" during a CDF-based table load

During long backfills of Change Data Feed (CDF) tracked tables, the native Delta Sharing native kernel can open more file descriptors simultaneously than the OS allows. Note: This affects macOS environments where the default limit is 256.

Raise the open files limit before starting the run: ulimit -n 65536.

  • macOS/Linux Shell: Add this command to your shell profile (~/.zshrc or ~/.bashrc) or to your scheduler's wrapper script.

  • Linux (systemd): Set LimitNOFILE=65536 in your systemd unit file.

  • Windows: No action needed; native OS limits are sufficient.

First run after upgrade takes much longer than usual

Expected behavior. The pipeline is processing your historical backfill data volume.

  • Monitor the Analytics Cloud Connector CLI logs.

  • The Analytics Cloud Connector CLI logs K to load partition counts; watch these numbers decrease to verify processing progress.

First run after upgrade shows fewer historical rows than expected

The processing_log row watermark was not cleared before execution, causing the Analytics Cloud Connector CLI to only load partitions newer than your old watermark.

Check for an existing watermark by running the following query:

  • SELECT * FROM <YOUR_DB>.<YOUR_SCHEMA>.PROCESSING_LOG
    WHERE TABLE_NAME = 'user_activity';

If a row for user_activity is returned, delete that row from the table and re-run the CLI execution to force the historical backfill.

Snowflake "out of memory" or warehouse size too small

The massive volume of historical data during a historical backfill exceeds your steady-state warehouse configuration.

Temporarily scale your Snowflake warehouse up for the duration of the backfill. For example, from X-Small to Medium or Large. Scale back down once completed.

Partial failure: some partitions loaded, then the run errored

Network drops or unexpected runtime interruptions.

Re-run the CLI script. It will scan your destination database, identify which daily partitions successfully landed, and pick up exactly where it left off without duplicating data.

"Corrupt footer" or Parquet errors during load

Package or dependency version mismatch, typically caused by mixing custom local delta-sharing or pyarrow installations.

Ensure you use the exact Python version and requirements.txt bundled with the new CLI. The best resolution is to utilize the bundled virtual environment (such as snowenv/).

Job exited with a non-zero code, but most tables succeeded

Expected behavior in the Analytics Cloud Connector CLI. Any individual table failure now forces the entire wrapper to exit with code 1 so schedulers accurately detect failures.

Review the Transfer Results summary printout at the end of the log to isolate the failing table:

  • Transfer Results:
    projects: Success (...)
    user_activity: Failed — <error>
    Job FAILED: 1 of N table(s) failed (user_activity). Exiting with code 1.

Resolve that specific table's issue, and re-run the job. Successful tables will skip ahead quickly.

Cannot locate the processing_log table

N/A

The location depends entirely on your connector type:

  • Snowflake: Located in the database and schema configured in your config.yaml file (the same location as your data tables).

  • SQL Server: Located in the schema configured in your config.yaml file.

  • Other Connectors: Located alongside your data files in the designated control/metadata area used by that specific connector.

Note: If it doesn't exist anywhere yet, the CLI will automatically generate it upon your first successful run.

See Also

Loading related articles...