hypequery.execution
Reference for the ClickHouse and embedded chDB executors.
Importing hypequery.execution does not import the driver or open a connection. The driver is loaded when you create an executor.
| Install | Gives you |
|---|---|
pip install "hypequery[clickhouse]==0.1.0b4" | The sync executor |
pip install "hypequery[clickhouse-async]==0.1.0b4" | The aiohttp-based async executor, and everything clickhouse gives |
pip install "hypequery[chdb]" | The embedded chDB executors (Linux and macOS; since 0.1.0b3) |
Executors
| Export | Purpose |
|---|---|
create_clickhouse_executor(connection) | A ClickHouseExecutor. Connects when called |
create_async_clickhouse_executor(connection) | An AsyncClickHouseExecutor running up to 8 queries at once. Connects when awaited |
create_async_from_sync_clickhouse_executor(connection, *, max_concurrent=4) | An AsyncFromSyncClickHouseExecutor running the sync driver on worker threads. Needs only the clickhouse extra and no aiohttp. Its admission queue belongs to the event loop it first waits on |
AsyncFromSyncClickHouseExecutor(client, control_client, *, max_concurrent=4) | A bounded worker bridge for apps that already have a synchronous driver client. Create both clients with autogenerate_session_id=False. aclose() waits for workers; close() closes without waiting, for shutdown without an event loop |
ClickHouseConnection(host="localhost", port=None, database="default", username="default", password="", secure=False, readonly_policy="query", control_username=None, control_password="") | Connection fields; passwords never appear in its repr |
ClickHouseConnection.from_env(environ=None, *, readonly_policy="query") | Read CLICKHOUSE_URL or CLICKHOUSE_HOST, _PORT, _DATABASE, _USERNAME, _PASSWORD, _SECURE (not both), plus _CONTROL_USERNAME and _CONTROL_PASSWORD |
ClickHouseConnection.from_url(url, *, readonly_policy="query", control_username=None, control_password="") | Read clickhouse://, clickhouses://, http(s)://, clickhousedb:// or clickhouse+http:// URLs. See Connecting |
Embedded chDB
| Export | Purpose |
|---|---|
create_chdb_executor(path=":memory:", *, database="default") | A ChdbExecutor on the chDB database at path, a directory or ":memory:". Unqualified sources resolve in database |
create_async_chdb_executor(path=":memory:", *, database="default") | An AsyncChdbExecutor running the same executor on one worker thread |
ChdbExecutor(connection) / AsyncChdbExecutor(connection) | Wrap a connection from chdb.connect() or a chdb.session.Session. close() closes it |
A chDB connection runs one query at a time and can't stop a running one. Its max_execution_time is shortened to the
caller's remaining time, in whole seconds with a one-second minimum. The async executor returns promptly on
cancellation or deadline expiry; a synchronous execute() waits until the running query finishes or reaches that
limit, which can be after the caller's deadline. A process holds one chDB path
at a time. See Embedded ClickHouse (chDB).
Pass the executor to create_dataset_client(executor=...) or create_async_dataset_client(executor=...).
Streaming
Every executor above also implements stream(compiled) and stream_arrow(compiled), which the clients' stream and
stream_arrow call. ClickHouse Arrow streams read the server's ArrowStream body and decode it as it arrives, so a
query that fails partway reports ClickHouse's error rather than a broken connection. A custom executor streams by
implementing StreamingQueryExecutor (or AsyncStreamingQueryExecutor) from hypequery.datasets.client; a client
given an executor without it raises TypeError from stream.
Results and read-only policy
QueryRows is the row type executors return.
ReadonlyPolicy is "query" or "profile". In query mode, a user the driver reports as already read-only (readonly 1 or 2) doesn't get the planner's readonly setting, because a readonly = 2 user would refuse it. Every other user gets all settings. profile mode does the same and also refuses a user that is not read-only.
A readonly = 1 user may not change any setting, so it refuses the planner's per-query limits and every query fails with
forbidden. Serve with a user whose profile sets readonly = 2: settings may change, data may not.
Cancellation and deadlines
A caller's cancellation, or an ExecutionContext deadline, stops the query on the server with
KILL QUERY … SYNC. The executor sends it on a separate, short-timeout control connection. Grant what that needs:
- By default the control connection signs in as the query user, which then needs
GRANT SELECT ON system.processes TO <user>. - With
control_username/control_passwordset (orCLICKHOUSE_CONTROL_USERNAME/_PASSWORD), cancellation runs as that user instead. It needsSELECT ON system.processesandKILL QUERY ON *.*, because it stops queries another user started.
If ClickHouse refuses the KILL, the executor logs one error on the hypequery.execution.cancellation logger and
stops asking. The query then runs until max_execution_time (30 s by default), though the caller still gets
aborted or deadline-exceeded immediately. The native async executor's driver also asks ClickHouse to drop a
read-only query when its request closes, when the user may change that setting.
await executor.aclose() stops any outstanding KILL retries, then closes the driver and control clients. The bridge
waits for its worker threads before closing. close() on the bridge stops admitting work without waiting.
Errors
Driver failures become a CompiledQueryError with a safe category; the driver's own text never reaches the caller.
| Category | When |
|---|---|
unauthenticated | ClickHouse refused the credentials |
forbidden | Access denied, or a readonly = 1 user refused the query settings |
not-found | The database does not exist |
too-large | The result exceeded its row or byte limit |
deadline-exceeded | The query timed out |
unavailable | ClickHouse could not be reached |
internal | Anything else |