Datasources.

Datasources

What it is. A governed connection to one of your databases. Every query a user runs against it passes through AccessFlow's review, masking, and row-security guards instead of hitting the database directly — so a datasource is where you decide who may run what against which data. PostgreSQL, MySQL, MariaDB, Oracle, and MS SQL Server are built in; MongoDB, Couchbase, Redis, Cassandra / ScyllaDB, Elasticsearch / OpenSearch, DynamoDB, Neo4j, Snowflake, BigQuery, and Databricks install from the connector catalog; any other JDBC engine works by uploading its driver and choosing Custom.

What is a datasource in AccessFlow?

A datasource is a governed connection to one of your databases. Users never receive its credentials — they submit queries to AccessFlow, which reviews them and then executes them over the pooled connection on their behalf. Masking, row-level security, schema allow-lists, and row caps are all configured per datasource.

Configure it. Create one with the four-step wizard at /datasources/new:

1. Database type. Pick a bundled driver tile (PostgreSQL ships built-in; other drivers download on first use and are verified against a pinned SHA-256 checksum). Pick Custom to use a JDBC driver you uploaded under Admin → Custom JDBC drivers.
2. Connection details. Name the datasource, then enter host, port, database name, service-account username, and password. SSL mode defaults to REQUIRE; switch to VERIFY_FULL in production. For Cassandra and ScyllaDB the wizard also requires a local datacenter (the driver's load-balancing datacenter); this field is unused for every other engine. For Elasticsearch and OpenSearch the wizard offers an Authentication toggle — basic (username + password) or an API key — and the database-name field is optional. For Amazon DynamoDB the connection is cloud credentials, not host/port: the wizard hides host/port and instead asks for the AWS region (the database-name field), the access key ID and secret access key (the username/password fields), and an optional custom endpoint (DynamoDB Local / VPC; blank for AWS). For Neo4j the wizard takes the standard host/port/database/username/password (the SSL mode is encoded in the Bolt scheme) plus an optional Bolt connection URI (advanced) — a full bolt:// / neo4j+s:// URI for Neo4j Aura or clustered routing that, when set, overrides host/port. For Snowflake the wizard asks for the account host (<account>.snowflakecomputing.com; the port field is hidden — always 443), the database, the user, a credential that is either a password or an unencrypted PKCS#8 private key (PEM) for key-pair authentication, and an optional JDBC URL override — a full jdbc:snowflake:// URL carrying warehouse / role / schema parameters. For Google BigQuery the connection is cloud credentials: the wizard hides host/port/username and asks for the GCP project (optionally project.dataset to pin a default dataset) and the service-account key JSON, plus an optional custom endpoint (BigQuery emulator). For Databricks SQL the wizard asks for the workspace host, the required warehouse HTTP path (/sql/1.0/warehouses/<id> from the warehouse's connection details), an optional Unity Catalog catalog, and a personal access token. The password (and API key / secret access key) are AES-256-GCM encrypted on write, decrypted once into the connection pool, and never returned in any GET response. When an external secrets manager is enabled (see Run & deploy), any credential field also accepts a secret reference — vault:<mount>/<path>#<field>, aws:<name-or-arn>[#jsonField], or azure:<secret-name> — stored as-is and resolved through the store at connection time; the form shows the syntax hints for whichever providers are enabled.
3. Connection test. AccessFlow opens a real JDBC connection, runs a heartbeat query, and surfaces any SSL / authentication errors before you save.
4. Configuration. Pick the Review plan that gates this datasource, toggle Require review on reads / writes, and (optionally) enable AI analysis and/or text-to-query + pick an AI configuration. The AI configuration is shared by both features, so it is required whenever either toggle is on. With text-to-query on, users can draft a query from a natural-language prompt in the editor — in the engine's native query language (SQL or a NoSQL query) — and the draft still flows through the normal review pipeline. Pool size, max rows, and statement timeout default sensibly but can be tightened per datasource.

Read replicas & load balancing (optional). On the datasource settings page, the Read replicas card takes any number of replica endpoints (JDBC URL plus optional username and password per endpoint — blank credentials reuse the primary's). AccessFlow opens one connection pool per endpoint and load-balances every query classified as SELECT round-robin across the healthy replicas; INSERT / UPDATE / DELETE / DDL and transactional BEGIN … COMMIT batches always hit the primary. Replicas must use the same database engine as the primary (they reuse the primary's JDBC driver), and credentials are AES-256-GCM encrypted with the same ENCRYPTION_KEY. Per-node health checks (a background prober plus a circuit breaker) take a failed endpoint out of rotation for a cooldown (ACCESSFLOW_PROXY_REPLICA_COOLDOWN, default 30s) and its health shows on the Datasource health dashboard; only when every replica is down does the read fall back to the primary, with one DATASOURCE_REPLICA_FALLBACK audit row visible at /admin/audit-log. Click Test replica on any row to validate its URL + credentials live without persisting; leaving the password blank reuses that endpoint's saved password. Remove every endpoint to disable replica routing. Replica pools reuse the same ACCESSFLOW_PROXY_* connection-pool tuning as the primary; the health checks are tuned by the ACCESSFLOW_PROXY_REPLICA_* variables.

SELECT result caching (optional). The settings page's Performance card opts a datasource into a Redis-backed result cache for repeated identical SELECTs, with a per-datasource TTL (1–86,400 seconds; blank uses ACCESSFLOW_PROXY_CACHE_DEFAULT_TTL, default 60s). Caching is security-safe by construction — entries are keyed over the row-security-rewritten query and the caller's masking scope, so masking and row-level security always apply — and any write executed through AccessFlow to a referenced table (including GDPR erasure and retention deletes) immediately invalidates the affected entries. Note that writes made outside AccessFlow are invisible to the cache and are served stale until the TTL expires, so pick a TTL that matches how the datasource is written. ACCESSFLOW_PROXY_CACHE_ENABLED=false switches the feature off deployment-wide.

Grant a user access. Open the datasource → Permissions tab and add a row per user — can read / can write / can DDL, allowed schemas, allowed tables, and restricted columns (masked as *** in SELECT results). Without a permission row, a user can't see or query the datasource at all. The allowed schemas / allowed tables lists are enforced when a query is submitted: every table it references — across joins, subqueries, CTEs, and BEGIN; …; COMMIT; batches — must appear in allowed tables or live in an allowed schema, or the query is rejected before it runs. Matching is case-insensitive, and an unqualified table name (FROM users) only matches an unqualified entry in allowed tables. Leave both fields empty to allow every table.

Schema explorer & ER diagram. Each datasource also carries Schema and ER diagram tabs alongside Configuration / Permissions. The schema view introspects the live database (cached and refreshable from the UI) and renders a searchable object tree — one filter matches across schema, table, and column names. Click any table to open a sample-data preview: a small, read-only set of rows fetched through the same governance path as a real query, so row-level security filters the rows and column masking redacts sensitive values (masked columns show ***, never the raw value). The same searchable tree and preview are available in the query editor sidebar. The ER tab lays those tables out as a node-and-edge graph with PK/FK badges and column types so reviewers and operators can sanity-check what a query is touching without leaving AccessFlow.

Masking policies. The datasource Masking tab adds per-column dynamic data masking on top of the static restricted-columns masking above. Each policy targets a schema.table.column and picks a strategy — full (***), partial (keep the last N characters), hash (stable SHA-256), email (j***@domain), or format-preserving — with an optional reveal-to condition. A query submitter whose role, group, or user id is listed in reveal to sees the unmasked value; everyone else sees the strategy output. A live preview shows how a sample value will render. Masking is applied at result-read time before results are serialized or stored, so unmasked values never persist, and the ids of the policies that applied are recorded in the execution's audit metadata. Reveal is explicit — there is no implicit admin bypass.

Row security policies. The datasource Row security tab adds row-level security: per-table predicates the proxy injects into the parsed SQL so a scoped user only sees (SELECT) or affects (UPDATE/DELETE) the rows they are authorised for. Each policy is a structured column operator value predicate where the value is a fixed literal or a :user.* variable — the built-in :user.id / :user.email / :user.role / :user.groups, or an admin-set per-user attribute (the Attributes key/value editor on Admin → Users). The applies to roles / groups / users scope it (empty = everyone, no implicit admin bypass — the inverse of masking's reveal to). Values are bound as parameters, never concatenated; an unresolved variable filters out every row (fail-closed); and a query the engine can't safely rewrite (a policied table inside a UNION, CTE, sub-select, or join-onto-another-policied-table) is rejected rather than run unfiltered. Applied policy ids are recorded in the execution's audit metadata.

Data classification

The datasource Classification tab tags tables and columns with one or more data classifications — PII, PCI, PHI, GDPR, FINANCIAL, or SENSITIVE — and derives stricter handling automatically. Tagging a column auto-applies a masking policy from the classification's default strategy (PII / GDPR / FINANCIAL → partial, PCI / PHI → full, SENSITIVE → hash), so you don't hand-configure masking for every sensitive field; a table-level tag (no column) is informational. A query that references a tagged table gets an automatic AI risk-score bump, and a derivation preview suggests a stricter review posture (AI review, human approval, minimum approvals) aggregated across the datasource's tags — a suggestion you apply on the datasource's review plan, never auto-changed. Tags are immutable (create / delete) and audited; deleting a tag keeps the masking policy it derived. The classifications appear as badges in the schema explorer, and Admin → Data classifications (/admin/data-classifications) lists every tag across all datasources as the evidence base for compliance reporting.

Automated discovery. Instead of tagging hundreds of tables by hand, the datasource Discovery tab opts a datasource into a scheduled scanner that samples column data through the same governed sampling path, detects sensitive values with local regex + checksum detectors (emails, credit-card numbers with Luhn, US SSNs, IBANs, phone numbers) and — optionally — your bound AI analyzer, then proposes the classification tags in a review worklist. Confirming a finding applies the tag (deriving masking exactly like a manual tag); dismissing suppresses the proposal permanently. Raw sampled values never persist (findings store a redacted sample only), and the AI pass only ever sees column names, types, and redacted samples. Configure the per-datasource sample size (10–1000 rows) and cadence (1–720 hours), or hit Scan now for an immediate run; scans and decisions land in the audit log (DISCOVERY_SCAN_COMPLETED, DISCOVERY_FINDING_CONFIRMED / _DISMISSED). Operator knobs: ACCESSFLOW_DISCOVERY_SCAN_POLL_INTERVAL (PT15M), ACCESSFLOW_DISCOVERY_SCAN_TIME_BUDGET (PT10M), ACCESSFLOW_DISCOVERY_SAMPLE_STATEMENT_TIMEOUT (PT10S), ACCESSFLOW_DISCOVERY_MAX_TABLES_PER_SCAN (200), ACCESSFLOW_DISCOVERY_MAX_AI_TABLES_PER_SCAN (25).

Tune it. Per-datasource fields above set row caps and review behaviour; these environment variables set the engine-level connection and execution ceilings (defaults shown):

• Connection pools (all JDBC engines): ACCESSFLOW_PROXY_CONNECTION_TIMEOUT (30s), ACCESSFLOW_PROXY_IDLE_TIMEOUT (10m), ACCESSFLOW_PROXY_MAX_LIFETIME (30m), ACCESSFLOW_PROXY_LEAK_DETECTION_THRESHOLD (0s = off).
• Statement execution (all engines): ACCESSFLOW_PROXY_EXECUTION_MAX_ROWS (10000), ACCESSFLOW_PROXY_EXECUTION_STATEMENT_TIMEOUT (30s), ACCESSFLOW_PROXY_EXECUTION_DEFAULT_FETCH_SIZE (1000), ACCESSFLOW_PROXY_EXECUTION_INSERT_BATCH_CHUNK_SIZE (1000).
• Heap protection (relational engines): ACCESSFLOW_PROXY_EXECUTION_MAX_RESULT_BYTES (52428800 — per-result byte cap; larger SELECT results are truncated), ACCESSFLOW_PROXY_EXECUTION_MAX_CONCURRENT (32 — global in-flight execution budget across all datasources), ACCESSFLOW_PROXY_EXECUTION_ACQUIRE_TIMEOUT (5s — wait before overflow executions are rejected with HTTP 503).
• SELECT result cache: ACCESSFLOW_PROXY_CACHE_ENABLED (true), ACCESSFLOW_PROXY_CACHE_DEFAULT_TTL (PT60S), ACCESSFLOW_PROXY_CACHE_MAX_ENTRY_BYTES (1000000).
• Read-replica health checks: ACCESSFLOW_PROXY_REPLICA_PROBE_INTERVAL (PT30S), ACCESSFLOW_PROXY_REPLICA_PROBE_TIMEOUT (PT5S), ACCESSFLOW_PROXY_REPLICA_COOLDOWN (PT30S).
• MongoDB: ACCESSFLOW_PROXY_MONGO_CONNECT_TIMEOUT (PT10S), …_SERVER_SELECTION_TIMEOUT (PT10S), …_MAX_POOL_SIZE (10).
• Couchbase: ACCESSFLOW_PROXY_ENGINES_COUCHBASE_CONNECT_TIMEOUT (PT10S), …_WAIT_UNTIL_READY_TIMEOUT (PT10S), …_SCAN_CONSISTENCY (request-plus).
• Redis: ACCESSFLOW_PROXY_ENGINES_REDIS_CONNECT_TIMEOUT (PT5S), …_SOCKET_TIMEOUT (PT5S), …_MAX_POOL_SIZE (10).
• Cassandra / ScyllaDB: ACCESSFLOW_PROXY_ENGINES_CASSANDRA_CONNECT_TIMEOUT / …_SCYLLADB_CONNECT_TIMEOUT (PT10S) and the matching …_REQUEST_TIMEOUT (PT10S).
• Elasticsearch / OpenSearch: ACCESSFLOW_PROXY_ENGINES_ELASTICSEARCH_CONNECT_TIMEOUT / …_OPENSEARCH_CONNECT_TIMEOUT (PT10S) and …_SOCKET_TIMEOUT (PT30S).
• DynamoDB: ACCESSFLOW_PROXY_ENGINES_DYNAMODB_CONNECT_TIMEOUT (PT10S), …_API_CALL_TIMEOUT (PT30S).
• Neo4j: ACCESSFLOW_PROXY_ENGINES_NEO4J_CONNECT_TIMEOUT (PT10S), …_MAX_CONNECTION_POOL_SIZE (100).
• Snowflake: ACCESSFLOW_PROXY_ENGINES_SNOWFLAKE_LOGIN_TIMEOUT (PT30S), …_NETWORK_TIMEOUT (PT60S).
• BigQuery: ACCESSFLOW_PROXY_ENGINES_BIGQUERY_CONNECT_TIMEOUT (PT10S), …_READ_TIMEOUT (PT30S).
• Databricks: ACCESSFLOW_PROXY_ENGINES_DATABRICKS_CONNECT_TIMEOUT (PT10S), …_WAIT_TIMEOUT (PT10S), …_POLL_INTERVAL (PT1S).

Custom JDBC drivers

What it is. The escape hatch for any JDBC database not in the connector catalog. Upload its driver JAR and datasources can bind to it by choosing Database type → Custom in the create wizard.

Configure it. Manage uploads at /admin/drivers:

1. Upload driver. Click Upload driver, attach the JAR, and supply the vendor name, target database type, the driver class identifier shown in the vendor's documentation, and the expected SHA-256 checksum.
2. Verification. The checksum is re-verified every time a pool initializes; uploaded drivers are loaded in isolated runtimes to keep providers from interfering with each other.
3. On-disk cache. JARs cache to ACCESSFLOW_DRIVER_CACHE (default ~/.accessflow/drivers). Mount it as a persistent volume in Kubernetes so pods don't re-download on restart, and set ACCESSFLOW_DRIVERS_OFFLINE=true for air-gapped installs.

Datasource health

What it is. An operational dashboard for spotting a datasource in trouble — pool exhaustion, a database that has turned slow, or a sudden volume spike. Use it as the first place to look when queries start failing or hanging.

Configure it. Nothing to set up. /admin/datasource-health (admin-only, read-only) auto-refreshes every 30 seconds; each card shows live connection-pool utilisation (active / idle / free against the configured maximum) plus a trailing 24-hour summary of query volume, p50 / p95 execution latency, and error count. Pool gauges read "pool not initialised" until the datasource's first query runs (pools are created on demand).

Tune it. ACCESSFLOW_PROXY_HEALTH_CACHE_TTL caches each snapshot so the auto-refresh stays cheap (default PT30S).