Infrastructure Management

Manage every Snowflake object through a visual interface. Terraform runs internally — customers never see HCL.

Snowflake Resource Management

• Databases — Create and manage Snowflake databases with naming conventions enforced. Configure data retention, transient/permanent options, and default warehouse assignment.
• Schemas — Organize data within databases. Create raw, staging, warehouse, and mart schemas aligned with your transformation methodology.
• Warehouses — Provision compute warehouses with size, auto-suspend, auto-resume, and scaling policy configuration. Monitor credit consumption per warehouse.
• Roles — Define Snowflake roles with hierarchy relationships. Role-to-role and role-to-user grants. Manage access centrally with Rime as the single source of truth.
• Users — Create and manage Snowflake users. Assign default roles, warehouses, and namespaces. Password and key-pair authentication support.
• Pipes — Configure Snowpipe for continuous data loading from S3. Auto-ingest with SNS notifications, pipe status monitoring.
• Stages — Manage external and internal stages for data loading. S3 integration with credential management.
• Tags — Apply Snowflake object tags for data classification and governance. Tag-based masking policy binding.
• Tag Associations — Bind tags to columns, tables, and schemas. Bulk tag assignment across objects.
• Masking Policies — Define column-level masking policies bound to tags. Full mask, partial mask (last 4 digits), or null replacement. Role-based visibility rules.

Grant Management

• Cascading dropdown selectors — Select databases, schemas, and objects from live Snowflake metadata. No manual typing of object names.
• 7 grant target types — Account objects, schemas, all schemas in database, future schemas, schema objects, all objects in schema, future objects in schema.
• Pattern-based grants — Use wildcards to grant access across objects matching a pattern. Automatically generates SHOW SCHEMAS LIKE data sources with for_each iteration.
• Multi-grant creation — Adding access to a schema automatically generates prerequisite USAGE grants on the parent database. One action, multiple resources.
• Privilege presets — Common privilege combinations (SELECT, INSERT, CREATE TABLE, etc.) grouped by object type.
• Futures support — Grant access to objects that do not exist yet. New tables and views in a schema automatically inherit the configured grants.
• Grant-by-default security — New roles have MANAGE GRANTS revoked by default. All grants flow through Rime. Optional override with a visible warning.

Infrastructure as Code

• Terraform under the hood — All Snowflake objects are managed by Terraform. Rime generates HCL from Handlebars templates, runs plan/apply in isolated workspaces, and manages state.
• Change preview — Before any changes are applied, see exactly what will be created, modified, or destroyed. Resource counts, attribute changes, and dependency impacts displayed in a clear summary.
• Plan/Apply workflow — Review planned changes before committing. No surprise modifications.
• Operation history — Full audit trail of every Terraform operation with timestamps, initiator, status, and output logs.
• Workspace isolation — Each project and environment gets its own Terraform workspace. No cross-contamination between tenants or projects.

AWS Resource Management

• S3 buckets — Provision landing zone buckets for data extraction with versioning, encryption, and lifecycle policies.
• SNS topics — Create notification topics for Snowpipe auto-ingest triggers.
• IAM roles and policies — Manage service roles with least-privilege policies for S3 access, Snowflake integration, and cross-account trust relationships.

Drift Detection

• Scheduled reconciliation — Rime periodically compares desired state (what you configured) against actual state (what exists in Snowflake). Runs automatically on a configurable interval.
• Drift alerts — When actual state diverges from desired state, resources are flagged as drifted. Integrate with alerting rules to get notified immediately.
• Remediation — One-click reapply to bring drifted resources back to desired state.

Declarative Configuration (Optional)

• Git-synced config repos — Optionally maintain desired state in YAML files in a Git repository. Rime watches for changes, parses the config, and reconciles infrastructure automatically.
• SSH and token auth — Connect to private Git repositories with SSH keys or access tokens.
• Branch tracking — Sync from a specific branch. Changes merged to the tracked branch trigger reconciliation.

Data Connectors

15 built-in connectors covering databases, SaaS platforms, DevOps tools, and file uploads. All built from scratch in Rust for performance and reliability.

Database Connectors

PostgreSQL

• Full schema discovery via information_schema
• Primary key detection for incremental loads
• Cursor-based extraction for memory-efficient large table reads
• Complete type mapping (text, integer, boolean, timestamp, numeric, UUID, JSON, arrays)
• SSL/TLS support

MySQL

• Schema discovery via INFORMATION_SCHEMA
• Primary key and index detection
• Batched Arrow RecordBatch extraction
• Full MySQL type mapping (VARCHAR, INT, DECIMAL, DATETIME, BLOB, ENUM, SET)
• Connection pooling via mysql_async

Microsoft SQL Server

• Schema discovery via INFORMATION_SCHEMA and sys.index_columns for primary keys
• Query-based extraction with configurable batch sizes
• Full MSSQL type mapping (NVARCHAR, INT, MONEY, DATETIME2, UNIQUEIDENTIFIER, XML)
• Windows authentication and SQL authentication support
• TDS protocol via tiberius

MongoDB

• Schema inference from document sampling (100 documents by default)
• BSON to Arrow type mapping (String, Int32, Int64, Double, Boolean, DateTime, ObjectId, Array, Document)
• Collection-level extraction with configurable filters
• Connection string authentication
• Replica set and sharded cluster support

Oracle

• Discovery via Oracle REST Data Services (ORDS)
• REST metadata catalog for table and column enumeration
• SQL-based extraction through ORDS query endpoints
• Bearer token authentication

SaaS Connectors

Salesforce

• OAuth2 password flow authentication
• SOQL query-based extraction
• Automatic pagination via nextRecordsUrl
• Standard and custom object support
• Field-level schema discovery

Xero

• Bearer token authentication
• 7 built-in objects: Contacts, Invoices, Bank Transactions, Payments, Accounts, Manual Journals, Items
• Page-based pagination with automatic traversal
• Rate limit handling

Shopify

• GraphQL Admin API integration
• Cursor-based pagination for large datasets
• Rate limit detection and backoff
• Products, orders, customers, inventory, and collection extraction

Google Sheets

• Sheets API v4 integration
• First-row header detection for automatic schema inference
• API key and access token authentication
• Multi-sheet support within a single spreadsheet

DevOps and DORA Connectors

Jira

• REST API v3 with JQL search
• Agile API for sprint data
• Changelog extraction for cycle time calculation
• Issues, sprints, worklogs, and transition history
• Cloud and Server/Data Center support

GitHub

• REST API with Link header pagination
• Rate limit handling with automatic backoff
• 5 objects: repositories, pull requests, commits, deployments, workflow runs
• Personal access token authentication

GitLab

• REST API v4 with keyset pagination
• Self-hosted instance support via custom base URL
• Projects, merge requests, pipelines, deployments, and environments
• Personal access token authentication

Azure DevOps

• WIQL query-based work item extraction
• Continuation token pagination for large result sets
• Batch work item fetching (up to 200 per request)
• Work items, pull requests, pipelines, releases, and builds

File Connectors

CSV

• Local file upload and parsing
• Configurable delimiter, quote character, and escape character
• Automatic schema inference from file contents
• Header row detection
• Large file support with streaming reads

JSON

• Local file upload and parsing
• Nested JSON flattening
• Array-of-objects and newline-delimited JSON (NDJSON) formats
• Schema inference from record sampling

REST API (Generic)

• Configurable HTTP client for any REST endpoint
• Multiple pagination strategies: page number, cursor, offset, link header
• Authentication: API key, Bearer token, Basic auth, OAuth2
• Custom headers and query parameters
• JSON response path extraction
• Rate limit detection

Connector SDK

• Trait-based architecture — All connectors implement a common Connector trait with test_connection(), discover_schema(), and extract() methods.
• Arrow-native — Extraction produces Apache Arrow RecordBatch streams for zero-copy interop with Parquet serialization.
• Schema discovery — Every connector discovers tables, columns, data types, primary keys, and nullable flags automatically.
• Connection testing — Validate credentials and connectivity before saving configuration. Immediate feedback on connection issues.
• Credential encryption — All credentials encrypted at rest with AES-256-GCM. Decrypted only at extraction time in the connector runner process.

Configuration

• 5-step wizard — Select connector type, configure connection details, test connectivity, discover and select tables, set sync schedule.
• Grouped connector catalog — Connectors organized by category: Databases, SaaS, DevOps/DORA, Files.
• Per-table selection — After schema discovery, select which tables to sync. Column-level selection available.
• Cron scheduling — Set sync frequency with standard cron expressions or preset intervals (hourly, daily, weekly).
• Sync history — View past sync runs with status, row counts, duration, and error details.
• Schema browser — Browse discovered tables and columns with data types, primary keys, and selection status.

Data Extraction

End-to-end pipeline from source systems to Snowflake. Configure once through the UI — Rime handles the rest.

Extraction Pipeline

• Arrow-native processing — Data extracted as Apache Arrow RecordBatches. Zero-copy columnar format enables efficient serialization and type preservation from source to destination.
• Parquet output — RecordBatches serialized to Parquet files with Snappy compression. Columnar storage optimized for analytical queries in Snowflake.
• S3 landing zone — Parquet files uploaded to a dedicated S3 bucket per project. Supports both AWS S3 and S3-compatible storage (MinIO for development and on-premise deployments).
• Snowpipe auto-ingest — S3 event notifications trigger Snowpipe to load data automatically. No manual COPY INTO statements. SNS topic integration for immediate ingestion.
• Per-table error handling — If one table fails during extraction, the remaining tables continue. Partial sync results are preserved. Failed tables are flagged with error details for retry.
• Batched extraction — Large tables extracted in configurable batch sizes. Memory-efficient streaming prevents OOM on tables with millions of rows.

Schema Discovery

• Automatic detection — Connectors discover all available tables, columns, data types, primary keys, and nullable constraints directly from the source system.
• Column-level metadata — Data types mapped from source-specific types (e.g., NVARCHAR(MAX), JSONB, DECIMAL(18,2)) to Arrow types and then to Snowflake types.
• Primary key identification — Primary keys detected from source system metadata. Used for incremental load configuration and data modeling downstream.
• Source schema storage — Discovered schemas persisted in Rime's database. Track schema changes over time without re-querying the source.
• Selective sync — After discovery, choose which tables and columns to sync. Only selected data is extracted and loaded.

Scheduling

• Cron-based scheduling — Standard cron expressions for flexible sync timing: every 15 minutes, hourly, daily at 2am, weekdays only.
• Timezone-aware — Schedules evaluated in the configured timezone with DST handling.
• On-demand sync — Trigger a sync manually at any time, independent of schedule.
• Due connector detection — Scheduler polls for connectors past their next scheduled sync and spawns extraction runs automatically.

Sync Monitoring

• Run history — Every sync run recorded with: start time, end time, duration, status (running, succeeded, failed), row counts per table, bytes written, and error messages.
• Real-time status — See which connectors are currently syncing and their progress.
• Error details — Failed runs include the specific error, which table caused the failure, and stack context for debugging.
• Row count tracking — Per-table row counts for each sync run. Compare across runs to detect volume anomalies.

Connector Runner

• Isolated execution — Each sync runs in a dedicated connector-runner process. Failures in one connector never affect others.
• Subprocess architecture — The API spawns connector-runner binaries with encrypted configuration. The runner decrypts credentials at runtime, executes the extraction, and reports results back to the database.
• CLI interface — The connector-runner binary accepts project ID, connector ID, and encryption key as arguments. Can be run manually for debugging.

Infrastructure Provisioning

• Automatic resource creation — When configuring extraction, Rime provisions the required AWS and Snowflake infrastructure: S3 bucket, SNS topic, IAM roles and policies, Snowpipe definition, and target schemas.
• Terraform-managed — All extraction infrastructure tracked and managed by the same Terraform workflow as other resources. No orphaned buckets or forgotten IAM policies.

Data Transformation

Generate and execute dbt models without writing SQL. Choose your methodology, configure sources and keys, and Rime builds the entire transformation layer.

Data Modeling Methodologies

Kimball (Dimensional Modeling)

• Staging models — Source-aligned cleaning layer. Generated from selected source tables with column selection and type casting.
• Dimension models — Slowly changing dimension tables with business keys, surrogate keys, and descriptive attributes. Configure grain and business key per dimension.
• Fact models — Transactional and periodic snapshot fact tables with measures, foreign keys to dimensions, and degenerate dimensions. Configure measures and grain.
• Mart models — Consumption-layer views joining facts and dimensions for analyst access.
• Available at all tiers.

Data Vault

• Hub models — Business entity tables with hash keys derived from business keys. Support for single and composite business keys (multiple columns concatenated with separator before MD5 hashing).
• Link models — Relationship tables connecting hubs with hash keys from participating hub keys. Many-to-many relationship support.
• Satellite models — Attribute storage tables attached to hubs or links. Hash diff for change detection. Load date tracking.
• Business vault — Derived business rules applied to raw vault data.
• Mart models — Consumption layer built on business vault.
• Available at Business tier and above.

Smart Data Modeling

• Source table discovery — Pull available tables directly from configured connectors. Tables grouped by connector with column metadata. No manual typing of table or column names.
• Column selection — Per-table multi-select of columns to include in models. Full column metadata (data type, nullable, primary key) displayed for informed selection.
• Model configuration wizard — 6-step guided flow: choose methodology, set project config (name, target database and schema), select source tables, select columns, configure models (assign tables as dimension/fact or define hubs/links/satellites), review and create.
• Automatic model generation — Based on your configuration, Rime generates the complete set of dbt models: staging, intermediate, and consumption layers with proper references, hash keys, and joins.

JSON Unpacking

• VARIANT column detection — When selecting columns, Rime identifies Snowflake VARIANT columns (common with API connector data) and offers JSON unpacking.
• Automatic path discovery — Runs LATERAL FLATTEN against a sample of rows to discover all nested keys, their data types, and sample values.
• Selective path extraction — Choose which JSON paths to include. Set aliases and cast types per path. Preview the expression that will be generated.
• Generated unpack models — Creates a dbt model that extracts selected paths from the VARIANT column with proper casting. References the staging model as upstream.

Hierarchy Models

• Parent-ID detection — Heuristic detection of columns containing "parent" and "_id" patterns, suggesting potential hierarchy relationships.
• Recursive CTE generation — Generates a dbt model with a recursive CTE that walks from root nodes (where parent_id is null) down to leaf nodes, building the full path at each level.
• Configurable separator — Set the path separator character (default: /). Path column concatenates labels from root to current node.
• Level tracking — Each row includes its depth level in the hierarchy, enabling level-based filtering and reporting.

Lineage Visualization

• DAG graph — Interactive lineage visualization powered by ReactFlow. See the full data flow from source tables through staging, intermediate, and consumption layers.
• Node details — Click any model node to see its name, type, materialization, SQL content, column list, and test results.
• Dependency tracking — Each model displays its upstream dependencies. Understand the impact of changes before making them.
• Manifest parsing — Lineage extracted from dbt's manifest.json after each run. Accurate, up-to-date dependency information.
• Cross-project lineage — Visualize data flows across multiple dbt projects. Detect shared tables between projects via database.schema.table matching. Available at Business tier and above.

dbt Execution

• Internal project generation — Rime generates complete dbt projects: dbt_project.yml, profiles.yml, source definitions, model SQL files, schema YAML with tests. Customers never see or manage these files.
• Run, test, and build — Trigger dbt run (execute models), dbt test (run data tests), or dbt build (both) from the UI.
• Run history — Every dbt execution recorded with start/end times, status, model counts (pass/fail/skip), test results, and full output logs.
• Result parsing — Run results extracted from run_results.json. Model-level pass/fail/error counts and test-level detail displayed in the UI.
• Snowflake integration — Profiles configured with Snowflake credentials (password or keypair authentication). Target database and schema configurable per project.

Templates

13 Handlebars templates covering all model types:

• dbt_project.yml — Project configuration
• profiles.yml — Snowflake connection (password and keypair auth)
• sources.yml — Source table definitions
• Staging, dimension, fact, and mart SQL templates (Kimball)
• Hub, link, satellite, and business vault SQL templates (Data Vault)
• JSON unpack and hierarchy SQL templates
• Schema YAML templates with column descriptions and tests

Pipeline Orchestration

Build end-to-end pipelines as visual DAGs. From extraction to validation in a single workflow.

DAG Engine

• Directed acyclic graph — Pipelines are modeled as DAGs where each step declares its dependencies. The engine validates the graph structure, detects cycles, and computes execution order using topological sorting.
• Parallel execution — Independent steps (no dependency between them) run concurrently up to a configurable concurrency limit. A pipeline with 3 independent extract steps followed by a transform step runs all 3 extractions in parallel.
• Dependency resolution — Steps only execute when all upstream dependencies have succeeded. If a dependency fails, downstream steps are automatically cancelled.
• Configurable concurrency — Set max concurrent steps per pipeline and max concurrent pipelines per tenant. Prevent resource contention during peak loads.

Step Types

• Extract — Run a configured connector to pull data from a source system. References an existing connector by ID.
• Provision — Apply infrastructure changes via Terraform. Create or modify Snowflake objects, S3 buckets, or IAM roles as part of the pipeline.
• Transform — Execute dbt models. Run the full model set or a specific subset.
• Validate — Run dbt tests against transformed data. Gate downstream steps on test results.
• Custom SQL — Execute arbitrary SQL against Snowflake. Useful for one-off data fixes, table swaps, or custom logic not covered by dbt.
• HTTP Webhook — Call an external HTTP endpoint. Configurable URL, method, headers, and body. Useful for triggering external systems or notifications.
• Conditional — Branch execution based on the result of a previous step. Expression evaluation determines whether downstream steps run.

Visual Pipeline Editor

• Drag-and-drop builder — Add steps from a palette and connect them by dragging edges between nodes. Built on ReactFlow.
• Color-coded step types — Each step type has a distinct visual style for quick identification in complex pipelines.
• Inline configuration — Click a step to configure its parameters: which connector, which dbt project, SQL to execute, webhook URL, etc.
• Validation — Click Validate to check the pipeline for issues: cycles, missing configurations, invalid dependencies. Errors displayed inline on the affected nodes.
• Delete and reconnect — Remove steps with the Delete key. Remaining connections automatically update.

Scheduling

• Cron expressions — Standard 5-field cron format for flexible scheduling. Presets for common intervals (every hour, daily, weekly).
• Timezone support — Every schedule has an associated timezone. Cron evaluated against wall-clock time in that timezone.
• DST-safe execution — Daylight saving time edge cases handled explicitly:
  • Spring forward (clock skips): the run triggers at the next valid instant after the gap.
  • Fall back (clock repeats): the run triggers on the first occurrence only; the second is skipped.
  • No double-fires, no silent skips.
• All timestamps stored as UTC — Internal timestamps (run records, logs, step durations) are always UTC. Conversion to display timezone happens in the frontend.

Execution

• Run triggers — Start a pipeline manually, on schedule, or via API.
• Real-time progress — WebSocket connection delivers live step status updates. See steps transition from pending to running to succeeded/failed as they execute.
• Retry policies — Per-step configuration: max retry attempts and exponential backoff (capped at 30 minutes). Transient failures retry automatically without manual intervention.
• Cancellation — Cancel a running pipeline. In-progress steps complete; pending steps are cancelled.
• Run history — Every pipeline execution recorded with: trigger type (manual, scheduled, API), start/end times, overall status, and per-step timeline with duration and output.
• Step-level logs — Expand any step in a run to see its output log: Terraform plan output, dbt run results, SQL query results, webhook response, etc.

Versioning

• Immutable versions — Each edit to a pipeline creates a new version. Previous versions are preserved.
• Version history — Browse all versions of a pipeline with creation dates and step counts.
• Rollback — Revert to any previous version with one click. Creates a new version with the rolled-back configuration (no destructive history rewrite).
• Current version tracking — Only one version is active at a time. Scheduled runs always use the current version.

Leader Election

• Multi-replica safe — In Kubernetes deployments with multiple scheduler replicas, PostgreSQL advisory locks ensure only one replica evaluates schedules and triggers runs. No duplicate executions.
• Automatic failover — If the leader replica goes down, another replica acquires the lock and takes over scheduling duties.

Pipeline Settings

• Status toggle — Enable or disable a pipeline without deleting it. Disabled pipelines skip scheduled runs but retain configuration.
• Concurrency limit — Max concurrent runs for this specific pipeline (separate from tenant-level limit).
• Timezone — Display and scheduling timezone for this pipeline.
• Delete — Remove a pipeline and all its versions, steps, and run history.

Monitoring and Alerting

Unified monitoring across your entire data stack. One dashboard for Snowflake health, dbt results, pipeline status, and data quality.

Alert Rules

• Configurable conditions — Define rules with metric source, condition operator, and threshold. Example: "Snowflake warehouse credit usage > 100 in 24 hours."
• Condition operators — Greater than, less than, equal to, not equal to, greater than or equal, less than or equal. Each operator evaluates a numeric metric value against a threshold.
• Metric sources — Snowflake, dbt, connector, infrastructure, and custom. Rules target specific subsystems for focused monitoring.
• Severity levels — Info, warning, critical. Controls notification urgency and escalation behavior.
• Active/inactive toggle — Disable rules temporarily without deleting them. Useful during maintenance windows.
• Evaluation interval — Rules evaluated every 60 seconds against the latest collected metrics.

Alert Lifecycle

• Pending to firing — New alerts start as pending. Once the condition is confirmed (not a transient spike), they transition to firing.
• Deduplication — Alerts are fingerprinted using SHA-256 of (rule_id + resource_type + resource_id). Duplicate alerts within a time window are suppressed.
• Auto-resolution — If the condition that triggered an alert clears, the alert resolves automatically. No manual cleanup.
• Acknowledgement — Mark an alert as acknowledged to stop escalation without resolving the underlying issue.
• Manual resolution — Explicitly resolve alerts that require human judgment.

Volume Anomaly Detection

• Rolling baseline — Maintains a 30-day rolling average of data volume per table.
• Deviation threshold — Alerts when daily volume deviates more than 50% from the rolling average. Catches both unexpected drops (source system issues) and unexpected spikes (data quality problems).
• Per-table tracking — Each table monitored independently. A volume anomaly in one table does not affect thresholds for others.
• Hourly evaluation — Anomaly detection runs every hour to catch issues within the same business day.

Advanced Anomaly Detection

• Seasonal decomposition — Separates time-series data into trend, seasonal, and residual components. Identifies recurring patterns (e.g., lower weekend volumes) and removes them from anomaly evaluation.
• Trend detection — Linear regression over the observation window to identify upward or downward trends. Alerts when trends exceed configurable thresholds.
• Z-score analysis — Residuals (data minus trend and seasonal components) evaluated for statistical outliers. Points beyond 2 standard deviations flagged as anomalies.

Notification Channels

• Email — SMTP-based email notifications. Available at all tiers.
• Slack — Webhook-based Slack notifications with formatted alert messages. Available at Small Business tier and above.
• PagerDuty — Events API v2 integration with acknowledgement sync. Available at Business tier and above.
• Webhooks — Configurable HTTP endpoints for custom integrations. Set URL, headers, and payload template. Available at Business tier and above.
• Tier-gated access — Channel availability enforced by licensing tier. Free tier is email only; higher tiers unlock additional channels.
• Connection testing — Test each channel configuration before saving. Sends a test notification to verify delivery.

Escalation Policies

• Ordered steps — Define a sequence of notification actions. If an alert is not acknowledged within a time window, escalate to the next step.
• Delay configuration — Set the wait time between escalation steps (e.g., 5 minutes, 15 minutes, 1 hour).
• Multi-channel steps — Each escalation step can notify a different channel: email first, then Slack, then PagerDuty.
• Automatic escalation — Escalation engine checks every 5 minutes for alerts that have exceeded their current step's delay without acknowledgement.

Metric Collection

• Connector poller — Tracks connector run status, row counts, durations, and error rates.
• dbt poller — Tracks model execution times, test pass/fail rates, and run durations.
• Infrastructure poller — Tracks Snowflake warehouse utilization, pipe status, and resource health.
• 5-minute polling interval — Metrics collected every 5 minutes by default. Stored in DuckDB for time-series querying.

Dashboards

Alert Dashboard

• Summary cards showing alert counts by severity (critical, warning, info).
• Filterable alert table with status, severity, rule name, resource, and timestamps.
• Acknowledge and resolve actions inline.
• 10-second auto-refresh for real-time visibility.

Infrastructure Health

• Resource counts grouped by type and status.
• Recent Terraform operations with outcomes.
• Snowflake warehouse utilization and credit burn rate.

Data Quality

• dbt test results over time (pass/fail/error trends).
• Volume anomaly flags per table.
• Data freshness tracking (time since last successful sync per source).

Data Retention

• DuckDB analytics store — Time-series metrics stored in an embedded DuckDB database for fast analytical queries.
• Retention policies — Raw metrics retained for 7 days, hourly aggregates for 90 days, daily aggregates for 1 year.
• Automatic cleanup — Retention manager runs every 24 hours to purge expired data.

Data Governance

Masked by default. Every column is masked until explicitly classified and unmasked per role.

Masked-by-Default Security

• Inverted security model — Unlike traditional approaches that find sensitive data and mask it, Rime masks everything by default. Data is only visible once it has been explicitly reviewed, classified, and unmasked for specific roles.
• Tag-based masking — Uses Snowflake's native tag-based masking policies. Privacy classification tags are applied to columns; masking policies are bound to tags; role grants determine visibility.
• Untagged data stays masked — Any column without an explicit classification tag remains under the default mask. There is no "forgot to mask it" scenario.
• Requires Snowflake Enterprise Edition — Tag-based masking policies are a Snowflake Enterprise feature. Available at Business tier and above in Rime.

Data Classification

• Privacy levels — Four classification levels: Public, Internal, Confidential, Restricted. Each level maps to different masking behaviors per role.
• PII types — Specific PII classifications: email, phone, name, address, IRD number, NHI number, credit card, date of birth, and custom types.
• Classification sources — Track whether a classification was auto-detected, manually assigned, or confirmed by a reviewer.
• Column browser — Browse all columns across all schemas in a project. View current classification, PII type, and masking status.
• Bulk operations — Classify or reclassify columns in bulk. Filter by schema, table, or current classification status.

PII Auto-Detection

• 10 regex patterns — Detects common PII by content pattern: email addresses, phone numbers (including NZ +64 format), credit card numbers (with Luhn checksum validation), NZ IRD numbers (with mod-11 check digit validation), NZ NHI numbers (alpha prefix + 4 digit format), UUIDs, dates of birth, physical addresses, full names, and bank account numbers (NZ format).
• Column name heuristics — Supplements pattern matching by checking column names for terms like "email", "phone", "ssn", "dob", "address".
• Snowflake sampling — Scans actual column values by querying a sample from Snowflake. Not just metadata-based — catches PII hiding in generic column names.
• Confidence scoring — Each detection includes a confidence indicator. High confidence for pattern matches with validation (e.g., Luhn check); lower confidence for heuristic-only matches.
• Review workflow — Auto-detected PII is flagged as "detected" and requires human confirmation or rejection before masking policies are applied.
• On-demand and automatic — Run a PII scan manually for any project, or automatically during schema discovery when connectors sync new data.

Masking Policies

• Masking types — Full mask (replace with ***), partial mask (show last 4 digits), null replacement (return NULL), and no mask (full visibility).
• Role-based visibility — Each masking policy specifies which roles see which masking level. An analyst role might see partial masks while an admin role sees full data.
• Masking matrix — Grid view showing all roles vs. all classification levels with the configured masking type for each cell. Quickly audit who can see what.
• Preview — Before applying, preview what a specific role would see for any table. Verify that masking policies produce the expected output.
• Apply to Snowflake — One-click deployment of masking policies, tags, and tag associations to Snowflake via Terraform. Rime generates the HCL and applies it.
• Policy templates — Tags, tag associations, and masking policies all managed as Terraform resources with Handlebars templates.

Snowflake RBAC

• Role hierarchy management — Define parent-child role relationships. Visualize the full role tree from account admin down to read-only roles.
• Grant management — Configure grants through cascading Snowflake-populated dropdowns. Select database, then schema, then object — all populated from live Snowflake metadata.
• 7 grant target types — Account objects, schemas, all schemas in database, future schemas, schema objects, all objects in schema, future objects in schema.
• Pattern matching — Wildcard grants across objects matching a pattern (e.g., all schemas starting with "stg_").
• Futures — Grant access to objects that do not exist yet. New tables automatically inherit configured grants.
• Multi-grant automation — Adding access to a schema object automatically generates prerequisite USAGE grants on the parent database and schema.
• User management — Create and manage Snowflake users. Assign default roles, warehouses, and namespaces.
• Grant-by-default security — New roles created by Rime have MANAGE GRANTS revoked by default. All access flows through Rime as the single source of truth.

Compliance Reporting

• Masking coverage — How many columns are still under the default mask vs. explicitly classified. Track progress toward full classification.
• PII detection results — Summary of detected PII by type and schema. Unreviewed detections highlighted.
• Access matrix — Per-role breakdown of accessible data, grouped by classification level.
• Unmasked column inventory — List of every column that has been explicitly unmasked, grouped by role. Audit trail for who approved each unmasking.
• Audit summary — High-level counts of recent actions, user activity, and policy changes.
• Export — Generate compliance reports as CSV or JSON for external audit submissions.

Security and Compliance

Enterprise-grade security from authentication through to audit trails. Built for NZ and AU enterprise procurement and security review requirements.

Authentication

Single Sign-On (SSO)

• SAML 2.0 — SP-initiated and IdP-initiated flows. Upload SAML metadata XML or provide metadata URL. Attribute mapping for user fields (email, name, groups). Compatible with Azure AD, Okta, OneLogin, and any SAML 2.0 identity provider.
• OpenID Connect (OIDC) — Discovery URL-based configuration. PKCE for enhanced security. Authorization code flow with automatic token exchange. Compatible with Azure AD, Okta, Auth0, Google Workspace, and any OIDC-compliant provider.
• JIT user provisioning — Users authenticated via SSO are automatically created in Rime on first login. No manual user provisioning required. Existing users matched by email are linked to their SSO identity.
• SSO testing — Test your SSO configuration before enforcing it. Verify that authentication flows complete successfully.
• Available at Business tier and above.

Google OAuth

• OAuth 2.0 authorization code flow.
• One-click sign-in with Google accounts.
• Automatic user creation on first login.
• Available at all tiers.

Email and Password

• Argon2id password hashing (memory-hard, resistant to GPU and ASIC attacks).
• JWT access tokens (short-lived) with refresh tokens (long-lived).
• Session management with explicit logout and token revocation.
• Available at all tiers as a fallback for smaller customers or initial setup.

Encryption

• In transit — TLS 1.3 for all API communication. HTTPS enforced.
• At rest — AES-256-GCM encryption for all sensitive data stored in Rime's database: Snowflake credentials, connector passwords, API keys, SSH private keys. Nonce prepended to ciphertext; encryption key derived from application configuration.
• Snowflake credentials — Stored as encrypted BYTEA columns. Decrypted only at the moment of use (e.g., when a connector run executes or a dbt profile is generated).
• Key management — Encryption key configured per deployment. Support for key rotation.

Multi-Tenant Isolation

• Database-per-tenant — Each tenant gets its own PostgreSQL database. Complete schema isolation. No shared tables, no row-level filtering.
• Pool routing — Middleware extracts the tenant from the subdomain ({tenant}.rimedata.io), lazily creates and caches connection pools per tenant.
• Tenant provisioning — Creating a new tenant provisions a new database, runs all migrations, and seeds default roles and configurations.
• Right-to-deletion — Database-per-tenant isolation makes GDPR, NZ Privacy Act, and AU Privacy Act right-to-deletion straightforward: drop the tenant database.

Audit Logging

• Every action recorded — User actions across the platform are logged: project creation, resource changes, connector configuration, dbt runs, SSO events, governance changes, masking policy updates.
• Audit fields — Timestamp, user ID, IP address, resource type, resource ID, action type, before state (JSON), after state (JSON).
• Fire-and-forget — Audit logging never fails a request. If the audit log write fails, the error is logged internally but the user's action proceeds.
• Searchable and filterable — Filter audit logs by user, action type, resource type, and date range.
• Expandable detail — Click any audit entry to see the full before/after state diff.
• Pagination — Large audit logs paginated for performance.
• Export — Download audit logs as CSV or JSON for external compliance submissions.
• Tier-based retention — Free: 7 days, Small Business: 30 days, Business: 90 days, Business Critical: 1 year. Automatic cleanup of expired entries.

NZ Privacy Act 2020

• Data residency — Deploy Rime in AWS or Azure NZ regions to keep all data within New Zealand.
• Right to access — Audit logs provide a complete record of what data exists and who has accessed it.
• Right to deletion — Database-per-tenant isolation enables clean tenant deletion.
• Data minimization — Column-level selection in connectors ensures only necessary data is extracted.
• Breach notification readiness — Audit trails and alert rules can detect and document unauthorized access patterns.

Australian Privacy Act 1988

• Australian Privacy Principles (APPs) — Rime's architecture supports compliance with all 13 APPs governing how personal information is collected, used, disclosed, and stored.
• APP 1 (Open and transparent management) — Audit logging and data classification provide a clear record of what personal information is held, how it was collected, and how it is used.
• APP 3 (Collection of solicited personal information) — Column-level selection in connectors ensures only necessary personal information is extracted. Data minimization is enforced at the point of collection.
• APP 5 (Notification of collection) — Data classification and PII detection provide visibility into what personal information exists across all data sources, supporting notification obligations.
• APP 6 (Use or disclosure) — Masked-by-default column security ensures personal information is not disclosed beyond its intended purpose. Role-based access controls restrict visibility per role.
• APP 8 (Cross-border disclosure) — Data residency options (AWS/Azure AU regions) keep all data within Australia. Database-per-tenant isolation prevents cross-tenant data leakage.
• APP 11 (Security of personal information) — AES-256 encryption at rest, TLS 1.3 in transit, Argon2id password hashing, and database-per-tenant isolation meet APP 11 security requirements.
• APP 12 (Access to personal information) — Audit logs and data classification provide a complete record of what personal information is stored, enabling access requests to be fulfilled.
• APP 13 (Correction of personal information) — Full audit trails with before/after state tracking support correction workflows and record-keeping.
• Notifiable Data Breaches (NDB) scheme — Alert rules, anomaly detection, and comprehensive audit trails support detection, assessment, and documentation of eligible data breaches within the 30-day notification window.
• Data residency — Deploy Rime in AWS Sydney (ap-southeast-2) or Azure Australia East to keep all data within Australian jurisdiction.

NZISM Alignment

• Encryption standards — AES-256 at rest, TLS 1.3 in transit meet NZISM encryption requirements.
• Access control — Role-based access with SSO integration meets NZISM identity and access management guidelines.
• Audit trails — Comprehensive logging with configurable retention meets NZISM audit requirements.
• Data classification — Privacy level tagging (Public, Internal, Confidential, Restricted) aligns with NZISM information classification standards.
• Readiness for government customer security reviews.

Migration and Onboarding

Import your existing environment in a single guided wizard. No destroy-and-recreate — Rime adopts your resources in place.

Snowflake Account Import

• Full account scan — Rime connects to your Snowflake account and discovers everything: databases, schemas, tables, views, warehouses, roles, role grants, pipes, stages, file formats, streams, tasks, masking policies, and tags.
• SHOW command introspection — Uses Snowflake's native SHOW and DESCRIBE commands to build a complete resource graph with dependencies.
• Dependency-aware ordering — Resources imported in the correct order: databases before schemas, schemas before tables, roles before grants. No broken references.
• Selective import — Choose which resources to bring under Rime management. Everything else stays untouched.
• Conflict detection — Resources that conflict with Rime conventions (naming collisions, incompatible configurations) are flagged with suggested resolutions.
• Terraform state adoption — For each imported resource, Rime runs terraform import to adopt the existing object into Terraform state. No "destroy and recreate" cycle.
• Dry-run preview — See exactly what will be imported and what the resulting Terraform state will look like before committing any changes.

Cloud Account Import (AWS)

• AWS resource scanning — Scans your AWS account for data infrastructure: S3 buckets (landing zones, staging areas), IAM roles and policies (pipeline service accounts), SNS topics, and related resources.
• Service-scoped permissions — Requires read access to S3, IAM, and SNS for the scan phase. Write access only needed for the resources Rime will manage going forward.
• HCL generation — Discovered cloud resources converted to Terraform HCL using the same template system as new resource creation.
• Terraform import — Each discovered resource adopted into Terraform state. Rime takes over lifecycle management without disrupting existing infrastructure.
• Azure and GCP ready — Architecture supports Azure (Storage Accounts, Data Factory, IAM) and GCP (Cloud Storage, BigQuery, Pub/Sub) scanning. AWS implemented first.

dbt Project Import

• Git repo connection — Connect to your existing dbt project's Git repository. SSH key and token authentication supported.
• Full project parsing — Parses dbt_project.yml, model SQL files, schema YAML files, sources.yml, and packages.yml.
• SQL analysis — Extracts ref() and source() calls, Jinja config blocks (materialization, tags, schema), and raw SQL from each model file.
• Methodology detection — Heuristically classifies existing models into Rime's model types: stg_ prefix and source() references map to Staging; dim_ prefix maps to Dimension; fct_ prefix maps to Fact; Data Vault naming patterns map to Hub, Link, Satellite; others default to Mart.
• Override classifications — Review auto-detected types and override any misclassifications before import.
• Lineage extraction — If target/manifest.json exists, uses pre-built lineage. Otherwise derives lineage from ref() and source() parsing.
• Validation — Flags models that reference sources not present in Rime's configured connectors. Suggests resolution paths.
• Incremental import — Re-run the import to pick up new models added to the repo since the last import.

Migration Wizard

An 8-step guided flow that can be paused and resumed at any point:

• 1. Connect — Provide Snowflake credentials (or select an existing connection) and AWS account credentials. Test connectivity for both.
• 2. Scan — Rime scans the Snowflake account and cloud account in parallel. Progress indicator with object counts per service.
• 3. Review Infrastructure — Tree view of discovered resources grouped by service: Snowflake (databases > schemas > tables, warehouses, roles) and AWS (S3 buckets, IAM roles). Checkboxes for selective import. Conflict warnings highlighted.
• 4. Import Infrastructure — Progress bar as Terraform imports run. Per-resource status: imported, skipped, or failed. Retry for any failures.
• 5. Connect dbt — Provide Git URL and credentials (or select existing config repo). Clone and parse.
• 6. Review Models — List of discovered models with auto-detected types. Override classifications. Lineage preview. Unresolvable reference warnings.
• 7. Import Models — Create DbtProject and DbtModel records. Optionally trigger a validation dbt run.
• 8. Summary — Everything imported, with links to the infrastructure inventory and model project pages.

The wizard is re-entrant: stop partway through and resume where you left off. Infrastructure and dbt imports are independent and can be done separately.

Guided Onboarding (New Customers)

A simplified 6-step flow for customers starting fresh:

• 1. Welcome — Introduction to Rime and what to expect during setup.
• 2. Create Project — Name your first project and set basic configuration.
• 3. Connect Snowflake — Enter Snowflake credentials and test the connection.
• 4. Add Connector — Configure your first data source. Select connector type, enter credentials, discover schemas.
• 5. Configure Models — Select source tables and configure initial transformation models.
• 6. Done — Setup complete. Links to next steps: create a pipeline, configure monitoring, set up governance.

Onboarding state is tracked per tenant. Progress persists across sessions. Steps can be skipped and revisited.

Migration State Management

• Database-tracked progress — Migration state stored in a dedicated table with step tracking and completion status. Survives session interruptions.
• JSONB scan results — Discovered resources, import results, and error details stored as JSONB for flexible querying and display.
• Status tracking — Each migration progresses through states: started, scanning, scanned, importing, completed, or failed.
• Available at all tiers — Migration tooling is not tier-gated. Frictionless onboarding is critical for adoption.

DORA Metrics

Measure engineering productivity with the four key metrics from the Accelerate research (Forsgren, Humble, Kim). Connect your DevOps tools and get automatic classification as Elite, High, Medium, or Low performing.

The Four Metrics

Deployment Frequency

How often your team deploys to production. Measured as deployments per day or per week. Calculated from deployment events in GitHub, GitLab, or Azure DevOps.

Lead Time for Changes

Time from code commit to production deployment. Measured in hours. Calculated as the duration from pull request open (or commit push) to merge to deployment. Shorter lead times indicate healthier delivery pipelines.

Mean Time to Restore (MTTR)

Time from incident detection to resolution. Measured in hours. Calculated from incident lifecycle events: open to resolved. Shorter MTTR indicates better operational resilience.

Change Failure Rate

Percentage of deployments that cause a failure requiring remediation. Calculated as rollback deployments divided by total deployments. Lower rates indicate higher deployment confidence.

Performance Classification

Metrics are automatically classified against the Accelerate research thresholds:

DevOps Connectors

Jira

• REST API v3 with JQL search for issue extraction
• Agile API for sprint data (velocity, burndown)
• Changelog extraction for calculating cycle time (in-progress to done)
• Issues, sprints, worklogs, and transition history
• API token authentication
• Cloud and Server/Data Center support

GitHub

• REST API with Link header pagination
• 5 extraction objects: repositories, pull requests, commits, deployments, workflow runs
• Lead time calculated as PR open to merge to deploy
• Change failure rate from rollback deployments vs. total deployments
• Rate limit handling with automatic backoff (5k/hr personal, 15k/hr enterprise)
• Some metrics (deployment frequency via Environments API, code scanning) require GitHub Team or Enterprise plan — Rime detects available features at connect time and surfaces what is accessible vs. gated

GitLab

• REST API v4 with keyset pagination
• Projects, merge requests, pipelines, deployments, and environments
• Similar metric calculation to GitHub
• Self-hosted instance support via custom base URL
• CI/CD analytics require GitLab Premium+

Azure DevOps

• WIQL query-based work item extraction
• Pull requests, pipelines, releases, and builds
• Boards analytics for cycle time and throughput
• Continuation token pagination for large result sets
• Batch work item fetching (up to 200 per request)

Common Data Model

All DevOps sources are normalized into a shared schema:

• Deployments — When, where, who deployed, success/failure
• Changes — Commits, pull requests, code modifications
• Incidents — Outages, failures, remediation events
• Recovery events — Incident resolution timestamps and actions

This normalization means DORA metrics are calculated consistently regardless of whether your team uses GitHub, GitLab, Azure DevOps, or a combination.

dbt Transformation

4 pre-built dbt model templates for DORA metric calculation:

• Deployments model — Aggregates deployment events, calculates frequency
• Changes model — Links commits to deployments, calculates lead time
• Incidents model — Tracks incident lifecycle, calculates MTTR
• Metrics model — Combines all sources, computes the four DORA metrics with Elite/High/Medium/Low classification per metric

DORA Dashboard

• 4 metric cards — Each displaying the current value and performance classification badge (Elite/High/Medium/Low).
• Trend visualization — Metric values over time to track improvement.
• Source connection status — Shows which DevOps tools are connected and contributing data. Prompts to connect additional sources for more complete metrics.

Available at Business tier and above.

Snowflake Cortex AI

Provision and manage Snowflake Cortex services directly from Rime. Full lifecycle management via SQL API.

Why Cortex through Rime

Snowflake Cortex services (Search, Analyst, Agents) lack full Terraform provider support. Rime provisions and manages them via Snowflake's SQL API, giving you the same clickops experience as other infrastructure resources with proper lifecycle tracking and status management.

Cortex Search Service

Full-text and semantic search over Snowflake tables.

• Configuration — Database, schema, warehouse, source table, search columns, attribute columns, and target lag.
• Search columns — Select which text columns to index for search queries. Multi-select from available table columns.
• Attribute columns — Select columns to return as metadata alongside search results.
• Target lag — Configure how frequently the search index refreshes from the source table.
• Provisioning — Rime generates and executes CREATE OR REPLACE CORTEX SEARCH SERVICE SQL with your configuration.
• Lifecycle — Create, monitor status (provisioning, active, error), and destroy search services through the infrastructure UI.

Cortex Analyst

Natural language queries over semantic models.

• Semantic model definition — Define your semantic model in YAML: tables, columns, relationships, measures, and dimensions.
• Internal stage management — Rime creates an internal Snowflake stage, uploads the semantic model YAML, and creates the Analyst reference.
• Configuration — Database, schema, warehouse, and the semantic model YAML content.
• Provisioning — Rime handles the multi-step process: stage creation, YAML upload, and model reference creation.
• Lifecycle — Full create/destroy management through the infrastructure UI.

Cortex Agent

Composite AI agents that combine multiple Cortex tools.

• Tool references — Configure agents with references to existing Cortex Search Services and Analyst models. Each tool has a type (cortex_search or analyst), a display name, and a service reference.
• Tool list builder — Add and remove tools from the agent configuration. Tools reference resources already provisioned in the same project.
• Configuration — Database, schema, warehouse, and the list of tools.
• Provisioning — Rime generates and executes CREATE OR REPLACE CORTEX AGENT SQL with tool specifications.
• Lifecycle — Full create/destroy management through the infrastructure UI.

SQL-Based Provisioning

Unlike Snowflake databases, warehouses, and roles (which Rime provisions via Terraform), Cortex resources use SQL-based provisioning because the Snowflake Terraform provider does not yet support them.

• CortexProvisioner — Dedicated provisioner that builds a Snowflake client, dispatches by resource type, and executes the appropriate SQL statements.
• Status tracking — Resources progress through statuses: Pending, Provisioning, Active (or Error). Status tracked in the same infrastructure resource table as Terraform-managed resources.
• Async provisioning — Cortex resource creation is spawned as an async task. The API returns immediately; the UI polls for status updates.
• Error handling — If provisioning fails, the resource is marked with Error status and the error message is stored for display in the UI.
• Destroy support — DROP ... IF EXISTS statements for clean resource removal.

Infrastructure UI Integration

Cortex resources appear alongside all other infrastructure resources in the infrastructure management page:

• Resource type labels — Cortex Search Service, Cortex Analyst, Cortex Agent displayed with readable names.
• Add Resource wizard — Cortex-specific configuration forms in the same Add Resource flow used for databases, schemas, and warehouses.
• Status indicators — Same Pending/Provisioning/Active/Error/Drifted status badges as other resources.
• Terraform exclusion — Cortex resources are automatically excluded from Terraform plan/apply operations. They appear in the resource list but are managed by the CortexProvisioner, not the TerraformJobRunner.