Skip to content

HQL pipes

The | operator is fundamental to HQL. A pipeline composes typed functions and should naturally support query and data-parallel execution. It remains a language design feature; the bootstrap parser does not accept pipelines yet.

For a unary function, the intended reading of x | f is f(x). A step consumes the previous value and returns the next. Input and output types must agree; piping a collection into a function expecting one Card is not implicitly a map.

cards
| typed RelationCard
| graph

This preferred example selects relation cards and requests a graph of them. cards is the reserved keyword for the namespace vault's cards; see card-type. typed RelationCard must check the contract, not the metadata.knowledge.type label alone; an authored entry does not establish conformance. The exact application syntax and the result type of graph remain design questions.

Functions as steps

A candidate composition style lets configured operations return unary functions:

people
| filter(p => p.age >= 18)
| map(p => p.name)

Here people is an explicitly supplied collection of typed values with age and name fields. Lambda spelling and partial application are illustrative. Argument placement, currying, shorthand fields, precedence and multiline continuation rules still need a grammar decision.

Prefer these ordinary functions over a trait system exposed to language users. That preference leaves room to design generic functions and type inference without importing Rust's syntax or extension mechanisms.

Execution freedom

Pure collection steps should leave room for lazy evaluation, step fusion, source-side query execution and partitioned work. These are executor options, not promises that every pipe runs in parallel or that a query planner exists.

An optimization must preserve observable values, sequence order and failures. Parallel reduction needs a suitable combining operation; arbitrary functions cannot be assumed associative. Sorting and whole-collection operations may require coordination or materialization. Effects must not be reordered or duplicated merely because they appear in a pipeline.

HQL-0004 gives this freedom a shape. x | f(a) is typed as f(x, a) and lowers to a stage the executor still sees; a chain of stages is a region. Every step declares, beside its purity, whether it is elementwise, a partition, a reduction with a known combiner, or a barrier, and that is the whole of what a backend may assume about splitting it. A checked program runs on a backend; localhost is the in-memory evaluator, one stage after another with no parallelism, and the oracle every other backend has to match. The reduce side — reduce, group, collect — is specified there and shown in the mapreduce example.

Unknown extension functions remain ordinary calls unless their declared contract allows more. Purity, failure and execution metadata, plus the eager/lazy boundary, are open design work shared with hql/extensions.

See hql/collections for ordering, hql/knowledge for terminal presenters, and card-type, doc-type and data-type for the values that flow through a pipeline.