Execution Paradigms Are Schedules

Sun, 19 Jul 2026 00:00:00 +0000
In the last post</a> I argued that the difference between a HyPer-style compiled pipeline and Inkfuse’s precompiled fragments comes down to which continuations get reified before runtime and which get inlined away. The post itself sparked a discussion with Alex Miller</a> about what an optimal IR for query compilers would look like, this discussion made me look at some old notes I wrote down a while ago while working on raptor</code>1</a></sup>, raptor</code> is a query compiler I started working on a while ago based on the same ideas as Lingo</a> it leverages MLIR</a> to build a tower of IRs as dialects. You first lower a logical plan to the relalg</code> dialect which maps closely to logical plan nodes, for example here’s what the aggregation operator looks like in MLIR’s tablegen constructs.</p>
class</span> RelAlg_Op</span><</span>string</span> mnemonic</span>,</span> list</span><</span>Trait</span>></span> traits</span> =</span> []</span>></span></span>
    :</span> Op</span><</span>RelAlg_Dialect</span>,</span> mnemonic</span>,</span> traits</span>></span>;</span></span>
</span>
def RelAlg_AggregationOp : RelAlg_Op</span><</span>"</span>aggregation</span>"</span>,</span> [</span></span>
    SingleBlockImplicitTerminator</span><</span>"</span>ReturnOp</span>"</span>></span></span>
]</span>></span> {</span></span>
    let summary </span>=</span> "</span>Group and aggregate</span>"</span>;</span></span>
    let description </span>=</span> [</span>{</span></span>
        Groups tuples by the grouping columns </span>and</span> computes aggregates.</span></span>
        The region specifies the aggregate computations.</span></span>
</span>
        Example:</span></span>
        %</span>1</span> =</span> relalg</span>.</span>aggregation</span> %</span>0</span></span>
            groupby </span>[</span>@</span>R</span>::</span>@key</span>]</span></span>
            [</span>#relalg</span>.coldef</span><</span>@</span>R</span>::</span>@total : i64</span>></span>]</span></span>
            (</span>%</span>tuple</span>: !</span>relalg</span>.</span>tuple</span>)</span> {</span></span>
            %</span>val </span>=</span> relalg</span>.</span>getcol</span> %</span>tuple @</span>R</span>::</span>@value : i64</span></span>
            %</span>sum </span>=</span> relalg</span>.</span>aggrfn</span> sum </span>%</span>val : </span>i64 </span>-></span> i64</span></span>
            relalg</span>.</span>return</span> %</span>sum : i64</span></span>
        }</span> : </span>!</span>relalg</span>.</span>tuplestream</span></span>
    }</span>]</span>;</span></span>
    let arguments </span>=</span> (</span>ins</span></span>
        RelAlg_TupleStreamType</span>:</span>$input</span>,</span></span>
        ArrayAttr</span>:</span>$groupByColumns</span>,</span></span>
        ArrayAttr</span>:</span>$computedColumns</span></span>
    )</span>;</span></span>
    let results </span>=</span> (</span>outs RelAlg_TupleStreamType</span>:</span>$result</span>)</span>;</span></span>
    let regions </span>=</span> (</span>region SizedRegion</span><</span>1</span>></span>:</span>$aggregates</span>)</span>;</span></span>
    let assemblyFormat </span>=</span> [</span>{</span></span>
        $input `groupby` $groupByColumns $computedColumns $aggregates</span></span>
        attr</span>-</span>dict `:`</span> type</span>(</span>$result</span>)</span></span>
    }</span>]</span>;</span></span>
}</span></span></code></pre>
The above dialect is then lowered to the suboperator dialect subop</code> which
you can think of as one tier lower.</p>
def SubOp_LoopOp : SubOp_Op</span><</span>"</span>loop</span>"</span>,</span> [</span></span>
    SingleBlockImplicitTerminator</span><</span>"</span>subop::YieldOp</span>"</span>></span></span>
]</span>></span> {</span></span>
    let summary </span>=</span> "</span>Loop over tuples with carry values</span>"</span>;</span></span>
    let description </span>=</span> [</span>{</span></span>
        Execute the body</span> for</span> each tuple</span>,</span> threading carry values through</span></span>
        the </span>loop</span>.</span> Used</span> for</span> aggregations </span>and</span> running computations.</span></span>
    }</span>]</span>;</span></span>
</span>
    let arguments </span>=</span> (</span>ins</span></span>
        SubOp_TupleStreamType</span>:</span>$input</span>,</span></span>
        Variadic</span><</span>AnyType</span>></span>:</span>$initCarry</span></span>
    )</span>;</span></span>
    let results </span>=</span> (</span>outs Variadic</span><</span>AnyType</span>></span>:</span>$finalCarry</span>)</span>;</span></span>
    let regions </span>=</span> (</span>region SizedRegion</span><</span>1</span>></span>:</span>$body</span>)</span>;</span></span>
</span>
    let assemblyFormat </span>=</span> [</span>{</span></span>
        $input `carry` `</span>(</span>` $initCarry `</span>:</span>`</span> type</span>(</span>$initCarry</span>)</span> `</span>)</span>` $body</span></span>
        attr</span>-</span>dict `:`</span> type</span>(</span>$input</span>)</span> `</span>-</span>></span>`</span> type</span>(</span>$finalCarry</span>)</span></span>
    }</span>]</span>;</span></span>
}</span></span>
</span>
def SubOp_BufferType : SubOp_Type</span><</span>"</span>Buffer</span>"</span>,</span> "</span>buffer</span>"</span>></span> {</span></span>
    let summary </span>=</span> "</span>A mutable buffer for storing tuples</span>"</span>;</span></span>
    let description </span>=</span> [</span>{</span></span>
        A buffer is a </span>mutable</span> container that can hold </span>tuples</span>.</span> Used</span> for</span></span>
        materializing intermediate results </span>and</span> building hash tables.</span></span>
    }</span>]</span>;</span></span>
    let parameters </span>=</span> (</span>ins ArrayRefParameter</span><</span>"</span>mlir::Type</span>"</span>></span>:</span>$columnTypes</span>)</span>;</span></span>
    let assemblyFormat </span>=</span> "</span>`<` $columnTypes `>`</span>"</span>;</span></span>
}</span></span>
</span>
def SubOp_HashIndexType : SubOp_Type</span><</span>"</span>HashIndex</span>"</span>,</span> "</span>hashindex</span>"</span>></span> {</span></span>
    let summary </span>=</span> "</span>A hash index over a buffer</span>"</span>;</span></span>
    let description </span>=</span> [</span>{</span></span>
        Provides hash</span>-</span>based access to tuples in a </span>buffer</span>.</span> The</span> key types</span></span>
        determine which columns are used</span> for</span> hashing.</span></span>
    }</span>]</span>;</span></span>
    let parameters </span>=</span> (</span>ins</span></span>
        ArrayRefParameter</span><</span>"</span>mlir::Type</span>"</span>></span>:</span>$keyTypes</span>,</span></span>
        ArrayRefParameter</span><</span>"</span>mlir::Type</span>"</span>></span>:</span>$valueTypes</span></span>
    )</span>;</span></span>
    let assemblyFormat </span>=</span> "</span>`<` `keys` `:` $keyTypes `,` `values` `:` $valueTypes `>`</span>"</span>;</span></span>
}</span></span></code></pre>
The final conversion step lowers subop</code> to scf</code></a>
which is another tier lower and can be lowered to actual LLVM IR which
you can ship off to Orc</a> to get machine
code.</p>
Of course as elegant as this looks there’s a lot of hidden details notably
you need to scaffold some sort of runtime that gives you all the nitty gritty
stuff your query execution code needs. For example you need to track lifetimes
of Arrow buffers that represent your columnar data around, you need to expose
bindings to some sort of allocator to handle buffers and so on.</p>
My theory on query execution is that if two engines that look nothing alike are the same program with different
decisions about where continuations live, then “vectorized” and “compiled” and “hybrid” are not rival architectures
so much as different points in some space, and it should be possible to name the axes of that space.</p>
The reason I am bringing this up is that my notes also referred to a concept I initially described as
sub-operator DAG + edge-located schedules</em> and since we can safely assume all operator DAGs are more
or less the same then the axes of the design space of query execution come down to what schedules you’re
picking or specializaing for.</p>
This post is an attempt to name these axes. The claim is that the execution paradigms we build systems
around are schedules</em> of one declarative suboperator program2</a></sup>, and that the schedule lives
on the edges of the plan rather than the nodes. But I also want to keep the claim from staying too abstract
so in this post I also describe a small enough compiler that represents the IR, schedules
a fragments library and a JIT. The full compiler can be found in schedules.py</a>.</p>
Charting the design space of query execution</h2>
The design space of query execution has mostly been charted by
building one system per point. MonetDB/X100 gave us vectorized
interpretation, HyPer gave us data-centric compilation and for a
decade they were treated as being separate approaches with different
trade-offs Kersten et al.’s comparison</a>
showed that which one wins depends mostly on the workload. Since then there has
been more hybrid designs like ROF</a> and
Inkfuse</a>.</p>
The most serious attempt to chart the space rather than add another point to it comes
from Gubner and Boncz VOILA</a> is a DSL
that describes what an operator does without pinning down the order or style in which
the operations run; from one VOILA program its backends can generate thousands of flavors</em>: vectorized,
data-centric, and many hybrids in between that nobody had bothered to hand-build.
The follow-up Excalibur</a>, takes the next
step: if there are thousands of flavors and no static cost model can robustly pick among
them, search at runtime. Excalibur encodes a point in the space as a sequence of
mutations, casts exploration as a multi-armed bandit problem, and switches flavors mid-query.</p>
Algorithms + Schedules</h2>
Let’s take a brief detour into the world of specialized compilers and talk about
Halide</a>. Halide split image pipelines into
an algorithm</em> which says what each stage computes and a schedule</em>, which says
where and when values are computed and stored: the loop order, the tiling, what gets fused into
its consumer and what gets materialized into a buffer. This approach allows you to turn
the problem from “rewrite the program until it is fast” into “search a space of schedules for a fixed program”
with the compiler guaranteeing that every schedule computes the same thing3</a></sup>.</p>
Query execution already has the algorithm half think back to Bandle and Giceva’s
sub-operator approach</a> and Jungmair and Giceva’s declarative
sub-operators</a> decompose relational
operators into small reusable pieces scatter,gather, scan, fold, hash-partition and so on
which stop prescribing how they execute a declarative suboperator DAG says which values
flow where and nothing else.</p>
In what follows we will mainly look at Q6 from TPC-H which is simple enough to write
a small compiler around but contains enough details to make this interesting.</p>
select</span> sum</span>(</span>l_extendedprice </span>*</span> l_discount) </span>as</span> revenue</span></span>
from</span> lineitem</span></span>
where</span> l_shipdate </span>>=</span> date</span> '</span>1994-01-01</span>'</span></span>
  and</span> l_shipdate </span><</span> date</span> '</span>1995-01-01</span>'</span></span>
  and</span> l_discount </span>between</span> 0</span>.</span>05</span> and</span> 0</span>.</span>07</span></span>
  and</span> l_quantity </span><</span> 24</span>;</span></span></code></pre>
We will represent suboperators as frozen dataclasses and the query plan will be just a list.</p>
</span>
#</span> Filter based on an operator technically we want to pass in a predicate</span></span>
#</span> but to keep things simple we will just pass-in some comparison operations.</span></span>
@</span>dataclass</span>(</span>frozen</span>=</span>True</span>)</span></span>
class</span> Filter</span>:</span></span>
    op</span>:</span> str</span>      #</span> "<", "<=", ">="</span></span>
    lhs</span>:</span> str</span>     #</span> column name</span></span>
    rhs</span>:</span> object</span>  #</span> int constant or column name</span></span>
</span>
@</span>dataclass</span>(</span>frozen</span>=</span>True</span>)</span></span>
class</span> Map</span>:</span></span>
    op</span>:</span> str</span>; </span>lhs</span>:</span> str</span>; </span>rhs</span>:</span> object</span></span>
    out</span>:</span> str</span>     #</span> name of the produced column</span></span>
</span>
#</span> Fold should allow arbitrary operations but again for simplicity we just assume</span></span>
#</span> summation.</span></span>
@</span>dataclass</span>(</span>frozen</span>=</span>True</span>)</span></span>
class</span> Fold</span>:</span></span>
    col</span>:</span> str</span>     #</span> running sum of this column</span></span>
</span>
#</span> This is the query plan as in sub-operator representation.</span></span>
Q6</span> =</span> [</span></span>
    Filter</span>(</span>"</span>>=</span>"</span>,</span> "</span>shipdate</span>"</span>,</span> 8766</span>)</span>,</span>   #</span> 1994-01-01</span></span>
    Filter</span>(</span>"</span><</span>"</span>,</span>  "</span>shipdate</span>"</span>,</span> 9131</span>)</span>,</span>   #</span> 1995-01-01</span></span>
    Filter</span>(</span>"</span>>=</span>"</span>,</span> "</span>discount</span>"</span>,</span> 5</span>)</span>,</span></span>
    Filter</span>(</span>"</span><=</span>"</span>,</span> "</span>discount</span>"</span>,</span> 7</span>)</span>,</span></span>
    Filter</span>(</span>"</span><</span>"</span>,</span>  "</span>quantity</span>"</span>,</span> 24</span>)</span>,</span></span>
    Map</span>(</span>"</span>*</span>"</span>,</span> "</span>extprice</span>"</span>,</span> "</span>discount</span>"</span>,</span> "</span>revenue</span>"</span>)</span>,</span></span>
    Fold</span>(</span>"</span>revenue</span>"</span>)</span>,</span></span>
]</span></span></code></pre>
Note what is absent. There are no loops, no batches, no buffers and no mention of code generation
instead everything is declarative and Q6</code> just says which values flow where and nothing else
which is what makes it an algorithm in Halide’s sense.</p>
scan(lineitem) ── e1 ──> filter ── e2 ──> map(price × disc) ── e3 ──> fold(sum)</span></span></code></pre>Location Scheduling</h2>
My proposal is that the schedule lives on those edges and we annotate each one with a tuple
composed of a data location</em> e.g. a register, a cache-resident buffer of some capacity, memory, a
representation</em> (dense vector, selection vector, on down to whatever your engine speaks)
and a granularity</em> (one tuple, a vector of 1024, a morsel). In code we can represent an annotation and a
schedule as:</p>
@</span>dataclass</span>(</span>frozen</span>=</span>True</span>)</span></span>
class</span> Edge</span>:</span></span>
    loc</span>:</span> str</span> =</span> "</span>buf</span>"</span>    #</span> "reg" | "buf"</span></span>
    rep</span>:</span> str</span> =</span> "</span>dense</span>"</span>  #</span> "dense" | "sel"</span></span>
</span>
@</span>dataclass</span>(</span>frozen</span>=</span>True</span>)</span></span>
class</span> Schedule</span>:</span></span>
    edges</span>:</span> tuple</span>        #</span> one Edge between each pair of adjacent ops</span></span>
    vector</span>:</span> int</span> =</span> 1024</span></span>
    tier</span>:</span> str</span> =</span> "</span>auto</span>"</span>  #</span> "auto" | "precompiled" | "jit"</span></span></code></pre>
We can then take the different paradigms we talked about before as just specializations and
describe them as functions:</p>
hyper</span> =</span> fuse_all</span>(</span>Q6</span>)</span>         #</span> every edge Edge("reg")</span></span>
vec</span>   =</span> stage_all</span>(</span>Q6</span>)</span>        #</span> every edge Edge("buf", "dense")</span></span>
rof</span>   =</span> stage_at</span>(</span>hyper</span>,</span> 4</span>)</span>   #</span> hyper with one edit</span></span></code></pre>
stage_at</code> is the most interesting one because it inserts a boundary by demoting an
edge from reg</code> to buf</code> which is what ROF does via pipeline buffers in between fused blocks.</p>
def</span> stage_at</span>(</span>sched</span>,</span> i</span>,</span> rep</span>=</span>"</span>dense</span>"</span>)</span>:</span></span>
    """</span>Demote one edge from reg to buf. ROF is one application of this.</span>"""</span></span>
    edges</span> =</span> list</span>(</span>sched</span>.</span>edges</span>)</span></span>
    edges</span>[</span>i</span>]</span> =</span> Edge</span>(</span>"</span>buf</span>"</span>,</span> rep</span>)</span></span>
    return</span> replace</span>(</span>sched</span>,</span> edges</span>=</span>tuple</span>(</span>edges</span>)</span>)</span></span></code></pre>
The paradigms that we’ve been describing end up being just schedule assignments to the plan
we drew above.</p>
hyper:      e1: reg        e2: reg        e3: reg</span></span>
vec:        e1: buf(1024)  e2: buf(1024)  e3: buf(1024)</span></span>
rof:        e1: reg        e2: buf(1024)  e3: reg</span></span></code></pre>
Here is what the annotations mean, as the code each row should compile down to. With
every edge in reg</code>, values flow between suboperators as local variables and the whole
pipeline is one loop:</p>
revenue</span> =</span> 0</span></span>
for</span> i</span> in</span> range</span>(</span>n</span>)</span>:</span></span>
    if</span> not</span> (</span>shipdate</span>[</span>i</span>]</span> >=</span> 8766</span>)</span>:</span> continue</span></span>
    if</span> not</span> (</span>shipdate</span>[</span>i</span>]</span> <</span>  9131</span>)</span>:</span> continue</span></span>
    if</span> not</span> (</span>discount</span>[</span>i</span>]</span> >=</span> 5</span>)</span>:</span>    continue</span></span>
    if</span> not</span> (</span>discount</span>[</span>i</span>]</span> <=</span> 7</span>)</span>:</span>    continue</span></span>
    if</span> not</span> (</span>quantity</span>[</span>i</span>]</span> <</span>  24</span>)</span>:</span>   continue</span></span>
    revenue</span> +=</span> extprice</span>[</span>i</span>]</span> *</span> discount</span>[</span>i</span>]</span></span></code></pre>
e1</code> and e2</code> appear nowhere: an edge in reg</code> is a value that is never stored anywhere
nameable the continuation between suboperators inlined, and it is what Hyper’s codegen
produces as LLVM IR.</p>
With every edge in buf</code>, each suboperator is its own loop and the edges are the lists between the loops:</p>
e1_price</span>,</span> e1_disc</span> =</span> [</span>]</span>,</span> [</span>]</span></span>
for</span> i</span> in</span> range</span>(</span>n</span>)</span>:</span>                    #</span> filter writes survivors into e1</span></span>
    if</span> passes_predicate</span>(</span>i</span>)</span>:</span></span>
        e1_price</span>.</span>append</span>(</span>extprice</span>[</span>i</span>]</span>)</span></span>
        e1_disc</span>.</span>append</span>(</span>discount</span>[</span>i</span>]</span>)</span></span>
e2</span> =</span> [</span>p</span> *</span> d</span> for</span> p</span>,</span> d</span> in</span> zip</span>(</span>e1_price</span>,</span> e1_disc</span>)</span>]</span>   #</span> map drains e1</span></span>
revenue</span> =</span> sum</span>(</span>e2</span>)</span>                                 #</span> fold drains e2</span></span></code></pre>
One benefit of these loops is that there is nothing specific to Q6; it can be compiled once
and reused by any query, which is similar to Inkfuse’s fragments. Do note that each boundary
introduces a copy which isn’t the case in the fused version.</p>
Inkfuse corresponds to the all-buf row, with the difference that its fragments are precompiled
instead of interpreted while Copy-and-patch keeps the boundary in registers: each stencil
ends with a tail call that passes its outputs in fixed registers to the next stencil, so the blocks
are precompiled but the edge between them is reg. This of course requires using the GHC calling convention
if you want to get a good idea about Copy-and-patch I recommend Alex’s article</a>.</p>
How do we define a region</em> ? Well we can define a region as the maximal tiling of sub-operators connected
by reg</code> edges so building a region is just about splitting the pipeline at buf</code> edges.</p>
def</span> regions</span>(</span>ops</span>,</span> sched</span>)</span>:</span></span>
    out</span>,</span> start</span> =</span> [</span>]</span>,</span> 0</span></span>
    for</span> i</span>,</span> e</span> in</span> enumerate</span>(</span>sched</span>.</span>edges</span>)</span>:</span></span>
        if</span> e</span>.</span>loc</span> ==</span> "</span>buf</span>"</span>:</span></span>
            out</span>.</span>append</span>(</span>(</span>start</span>,</span> i</span> +</span> 1</span>)</span>)</span></span>
            start</span> =</span> i</span> +</span> 1</span></span>
    out</span>.</span>append</span>(</span>(</span>start</span>,</span> len</span>(</span>ops</span>)</span>)</span>)</span></span>
    return</span> out</span></span></code></pre>
hyper</code> produces one region of seven suboperators, vec</code> produces seven regions of one, and rof</code> produces two
i.e the five filters fused, then map and fold fused. The actual fusion structure is not a property anyone
states we just derive it from where the buffers are.</p>
There is a fourth annotation that has no Halide analog and it applies to regions rather
than edges which I will call tiering</em>. A good way to think about tiering is how V8 for
example uses 4 tiers of compilers</a> depending on different heuristics.
We can think of tiering in our case not in the context of JIT thresholds or hotness but instead
where the code for a region comes from in the baseline case we interpret it out of a library of single-node primitives
or look it up in a library of precompiled multi-node fragments or compile it for this query.</p>
I think this allows us to answer the axis jamii called the missing tier for query compilers</a>,
in terms of pipeline boundaries. Inkfuse ties the two together: a fragment is defined by both
its boundary and the fact that it is precompiled. Keeping tier separate from the edge annotations means
you can change one without the other: swap a JIT region for an equivalent precompiled block without touching
the buffers (this is what a fragment cache does), or move a buffer without changing how the code is produced
which is what ROF does.</p>
Two pipelines and a join</h2>
Q6 is a single straight pipeline, which is enough to introduce the
coordinates but not enough to stress them the scheduling questions people
actually argue about show up around joins, so before building the
tiers it is worth seeing what a join looks like in this notation.
TPC-H Q14:</p>
select</span> 100</span>.</span>00</span> *</span> sum</span>(</span>case</span> when</span> p_type </span>like</span> '</span>PROMO%</span>'</span></span>
                         then</span> l_extendedprice </span>*</span> (</span>1</span> -</span> l_discount)</span></span>
                         else</span> 0</span> end</span>)</span></span>
             /</span> sum</span>(</span>l_extendedprice </span>*</span> (</span>1</span> -</span> l_discount)) </span>as</span> promo_revenue</span></span>
from</span> lineitem, part</span></span>
where</span> l_partkey </span>=</span> p_partkey</span></span>
  and</span> l_shipdate </span>>=</span> date</span> '</span>1995-09-01</span>'</span></span>
  and</span> l_shipdate </span><</span> date</span> '</span>1995-10-01</span>'</span>;</span></span></code></pre>
Decomposed into suboperators, Q14 is two pipelines. Jungmair’s IR and
Excalibur’s low-level plan both break a hash join apart in roughly
this way:</p>
build: scan(part) ── b1 ──> insert(ht)</span></span>
</span>
probe: scan(lineitem) ── e1 ──> filter(shipdate) ── e2 ──> probe(ht)</span></span>
         ── e3 ──> gather(p_type) ── e4 ──> map(promo?, revenue) ── e5 ──> fold(sums)</span></span></code></pre>
insert</code> builds a hash table keyed on p_partkey</code>. The probe pipeline
cannot start until the build has finished, so ht</code> is not an edge
between the pipelines; it is shared state, and the build/probe
dependency is what pipeline-oriented compilers call a pipeline
breaker. On the probe side, probe</code> looks up l_partkey</code> in the table
and gather</code> fetches p_type</code> out of the matched row. A monolithic
HashJoin operator does both steps internally. Splitting them apart
matters because the gather is a dependent load per matching row,
usually a cache miss, and once it is a separate node the schedule can
decide where it runs and on how many rows at a time.</p>
The edge to look at is e2</code>, between the filter and the probe. With
e2: reg</code> the probe is inlined into the scan/filter loop, which is
how a fully fused pipeline runs this query: each surviving row probes
the hash table immediately, and when the table is bigger than cache
the load stalls with nothing else in flight to hide it. With
e2: buf(1024)</code> the filter accumulates a vector of pending keys
before any probe runs, and a buffer of pending keys is the
precondition for software prefetching4</a></sup>: while probing
key i</code> you issue a prefetch for the bucket of key i + k</code>, and by
the time the loop reaches it the bucket is in cache. This was the
original motivation for ROF, and in this notation the paper’s stage
boundary in front of the probe is the annotation e2: buf</code>.</p>
The walkthrough compiler stops short of all of this: it handles
straight pipelines only, and Q14 is the first query that would force
a second pipeline, shared state, and a barrier into it. I’ll come
back to what that state costs the formalism at the end.</p>
Tier one: the block library</h2>
Back to Q6 and the compiler. The first tier we will need to stitch together
is some pre-compiled primitives. Because we don’t know what every query we
will be processing looks like we want to limit ourselves to a combination
of primitive operations, but how do we decide which combination of primitive
operations should we pre-compile ?</p>
The edge annotations answer this because they are exactly what identifies a
compiled block. A block is code for some fused region; its interior edges were
inlined away, so what survives is the boundary: which suboperators the region
computes, plus the representation of each edge crossing in or out. I call that
tuple the region’s boundary key</em>, and in the compiler it is a plain tuple:</p>
def</span> region_key</span>(</span>ops</span>,</span> sched</span>,</span> start</span>,</span> end</span>)</span>:</span></span>
    parts</span> =</span> tuple</span>(</span>shape_of</span>(</span>o</span>)</span> for</span> o</span> in</span> ops</span>[</span>start</span>:</span>end</span>]</span>)</span></span>
    rin</span>,</span> rout</span> =</span> in_rep</span>(</span>sched</span>,</span> start</span>)</span>,</span> out_rep</span>(</span>sched</span>,</span> end</span> -</span> 1</span>)</span></span>
    if</span> end</span> -</span> start</span> ==</span> 1</span>:</span></span>
        return</span> (</span>*</span>parts</span>[</span>0</span>]</span>,</span> rin</span>,</span> rout</span>)</span>     #</span> e.g. ("filter", ">=", "const", "dense", "sel")</span></span>
    return</span> (</span>"</span>fused</span>"</span>,</span> parts</span>,</span> rin</span>,</span> rout</span>)</span></span></code></pre>
Notice that the key doesn’t refer to explicit column names or constants ("filter", ">=", "const", ...)</code>
covers shipdate >= 8766</code> and quantity >= 5</code> alike, because those arrive at the block as call-time parameters.
The key just holds what the code shape depends on and the runtime state holds what this query binds to.</p>
The library itself is a dictionary from boundary keys to code:</p>
def</span> build_library</span>(</span>)</span>:</span></span>
    lib</span> =</span> {</span>}</span></span>
    for</span> sym</span> in</span> (</span>"</span>+</span>"</span>,</span> "</span>*</span>"</span>,</span> "</span><</span>"</span>,</span> "</span><=</span>"</span>,</span> "</span>>=</span>"</span>)</span>:</span></span>
        for</span> sh</span> in</span> (</span>"</span>const</span>"</span>,</span> "</span>col</span>"</span>)</span>:</span></span>
            for</span> rin</span>,</span> rout</span> in</span> (</span>(</span>"</span>dense</span>"</span>,</span> "</span>dense</span>"</span>)</span>,</span> (</span>"</span>dense</span>"</span>,</span> "</span>sel</span>"</span>)</span>,</span> (</span>"</span>sel</span>"</span>,</span> "</span>sel</span>"</span>)</span>)</span>:</span></span>
                lib</span>[</span>(</span>"</span>filter</span>"</span>,</span> sym</span>,</span> sh</span>,</span> rin</span>,</span> rout</span>)</span>]</span> =</span> make_filter</span>(</span>sym</span>,</span> rin</span>,</span> rout</span>)</span></span>
            for</span> rep</span> in</span> (</span>"</span>dense</span>"</span>,</span> "</span>sel</span>"</span>)</span>:</span></span>
                lib</span>[</span>(</span>"</span>map</span>"</span>,</span> sym</span>,</span> sh</span>,</span> rep</span>,</span> rep</span>)</span>]</span> =</span> make_map</span>(</span>sym</span>,</span> rep</span>)</span></span>
    for</span> rep</span> in</span> (</span>"</span>dense</span>"</span>,</span> "</span>sel</span>"</span>)</span>:</span></span>
        lib</span>[</span>(</span>"</span>fold</span>"</span>,</span> "</span>+</span>"</span>,</span> rep</span>,</span> None</span>)</span>]</span> =</span> make_fold</span>(</span>rep</span>)</span></span>
    #</span> no fused key is ever inserted</span></span>
    return</span> lib</span></span></code></pre>
Here is make_map</code>, trimmed of its constant-operand handling:</p>
def</span> make_map</span>(</span>sym</span>,</span> rep</span>)</span>:</span></span>
    fn</span> =</span> OP_FNS</span>[</span>sym</span>]</span></span>
    def</span> block</span>(</span>chunk</span>,</span> o</span>,</span> needed</span>)</span>:</span>      #</span> o.lhs, o.rhs, o.out bind at call time</span></span>
        lhs</span>,</span> rhs</span> =</span> chunk</span>.</span>cols</span>[</span>o</span>.</span>lhs</span>]</span>,</span> chunk</span>.</span>cols</span>[</span>o</span>.</span>rhs</span>]</span></span>
        out</span> =</span> [</span>0</span>]</span> *</span> chunk</span>.</span>n</span></span>
        for</span> i</span> in</span> indices</span>(</span>chunk</span>)</span>:</span>      #</span> chunk.sel, or range(chunk.n) if dense</span></span>
            out</span>[</span>i</span>]</span> =</span> fn</span>(</span>lhs</span>[</span>i</span>]</span>,</span> rhs</span>[</span>i</span>]</span>)</span></span>
        return</span> with_column</span>(</span>chunk</span>,</span> o</span>.</span>out</span>,</span> out</span>)</span></span>
    return</span> block</span></span></code></pre>
The dense and sel variants differ in their iteration space and in the contract of
their output, which is why representation is part of the key we have the same suboperator
but a different edge type and different code. In a native engine the difference will
manifest itself in the form of instruction density, auto-vectorization and cache misses
because the dense body will be branch-free and auto-vectorizes and the sel body will have
pointer and indices chasing and not always vectorize.</p>
Different representations require passing data across boundaries, a filter for example produces
its surviving rows as a selection vector and a downstream block that computes a property over
rows wants a dense vector. So we need a way to copy or coerce the data in any engine this will
be an implementation hidden detail and won’t appear as a plan-level construct but in our case we
want to have a conversion node that materializes selection vectors into fresh columns.</p>
@</span>dataclass</span>(</span>frozen</span>=</span>True</span>)</span></span>
class</span> Compact</span>:</span></span>
    """</span>sel -> dense: materialize the survivors a selection vector</span></span>
    names into fresh columns.</span>"""</span></span>
</span>
def</span> make_compact</span>(</span>)</span>:</span></span>
    def</span> block</span>(</span>chunk</span>,</span> o</span>,</span> needed</span>)</span>:</span></span>
        if</span> chunk</span>.</span>sel</span> is</span> None</span>:</span></span>
            return</span> chunk</span></span>
        cols</span> =</span> {</span>c</span>:</span> [</span>chunk</span>.</span>cols</span>[</span>c</span>]</span>[</span>i</span>]</span> for</span> i</span> in</span> chunk</span>.</span>sel</span>]</span></span>
                for</span> c</span> in</span> chunk</span>.</span>cols</span> if</span> c</span> in</span> needed</span>}</span></span>
        return</span> Chunk</span>(</span>cols</span>,</span> len</span>(</span>chunk</span>.</span>sel</span>)</span>,</span> None</span>)</span></span>
</span>
lib</span>[</span>(</span>"</span>compact</span>"</span>,</span> "</span>sel</span>"</span>,</span> "</span>dense</span>"</span>)</span>]</span> =</span> make_compact</span>(</span>)</span></span></code></pre>
Being an ordinary suboperator is the point. Compact</code> sits in the op
list, regions</code> splits around it like anything else, and it resolves
through the same dictionary lookup, so the conversion is a thing in the
plan rather than a thing inside somebody’s filter loop. compact</code> is
the one coercion the walkthrough implements. Two more belong to the
same family and are not in the code: gather fetches values through a
level of indirection (we met one fetching p_type</code> through Q14’s probe
result), and spill demotes a register region into a buffer.</p>
Speaking of legality a schedule can run on the precompiled tier if every one of its regions’s
keys is in the dictionary. plan</code> does that check while resolving each region to a callable:</p>
def</span> plan</span>(</span>ops</span>,</span> sched</span>,</span> lib</span>)</span>:</span></span>
    resolved</span> =</span> [</span>]</span></span>
    for</span> start</span>,</span> end</span> in</span> regions</span>(</span>ops</span>,</span> sched</span>)</span>:</span></span>
        key</span> =</span> region_key</span>(</span>ops</span>,</span> sched</span>,</span> start</span>,</span> end</span>)</span></span>
        if</span> key</span> in</span> lib</span> and</span> sched</span>.</span>tier</span> !=</span> "</span>jit</span>"</span>:</span></span>
            resolved</span>.</span>append</span>(</span>bind</span>(</span>lib</span>[</span>key</span>]</span>,</span> ops</span>[</span>start</span>]</span>)</span>)</span>        #</span> tier: precompiled</span></span>
        elif</span> sched</span>.</span>tier</span> ==</span> "</span>precompiled</span>"</span>:</span></span>
            raise</span> IllegalSchedule</span>(</span>f</span>"</span>{</span>key</span>}</span> is not in the library</span>"</span>)</span></span>
        else</span>:</span></span>
            resolved</span>.</span>append</span>(</span>jit_region</span>(</span>ops</span>,</span> sched</span>,</span> start</span>,</span> end</span>)</span>)</span>  #</span> tier: jit</span></span>
    return</span> resolved</span></span></code></pre>
There is no separate list of allowed schedules; the check is dictionary membership, so adding
a block to the library is what makes the schedules that need it legal. Another thing to notice
is that we don’t have to fix one convention for whether filters emit selection vectors or compacted
output. The same works for Q14, with one addition: the boundary key of a probe region also names the
hash table’s bucket layout, because a precompiled probe block commits to a bucket format the same way
a map block commits to a vector width.</p>
Physical Layouts as Schedule Decisions</h2>
Another thing that needs to be encoded is the physical layout of hashmap buckets
this only matters because our approach allows us to define an arbitrary layout
for prob/build</code> hashmaps and any storage interface.</p>
EMPTY</span> =</span> -</span>1</span></span>
</span>
#</span> (insert, layout="open") — open addressing with linear probing:</span></span>
#</span> two flat arrays, collisions resolved by walking to the next slot.</span></span>
def</span> build_open</span>(</span>keys</span>,</span> payloads</span>,</span> bits</span>)</span>:</span></span>
    mask</span> =</span> (</span>1</span> <<</span> bits</span>)</span> -</span> 1</span></span>
    ht_keys</span> =</span> [</span>EMPTY</span>]</span> *</span> (</span>1</span> <<</span> bits</span>)</span></span>
    ht_pay</span>  =</span> [</span>0</span>]</span> *</span> (</span>1</span> <<</span> bits</span>)</span></span>
    for</span> k</span>,</span> p</span> in</span> zip</span>(</span>keys</span>,</span> payloads</span>)</span>:</span></span>
        h</span> =</span> hash</span>(</span>k</span>)</span> &</span> mask</span></span>
        while</span> ht_keys</span>[</span>h</span>]</span> !=</span> EMPTY</span>:</span></span>
            h</span> =</span> (</span>h</span> +</span> 1</span>)</span> &</span> mask</span></span>
        ht_keys</span>[</span>h</span>]</span>,</span> ht_pay</span>[</span>h</span>]</span> =</span> k</span>,</span> p</span></span>
    return</span> {</span>"</span>keys</span>"</span>:</span> ht_keys</span>,</span> "</span>pay</span>"</span>:</span> ht_pay</span>,</span> "</span>mask</span>"</span>:</span> mask</span>}</span></span>
</span>
#</span> (insert, layout="chained") — bucket heads pointing into entry</span></span>
#</span> arrays, collisions resolved by a next-pointer chain.</span></span>
def</span> build_chained</span>(</span>keys</span>,</span> payloads</span>,</span> bits</span>)</span>:</span></span>
    mask</span> =</span> (</span>1</span> <<</span> bits</span>)</span> -</span> 1</span></span>
    heads</span> =</span> [</span>EMPTY</span>]</span> *</span> (</span>1</span> <<</span> bits</span>)</span></span>
    ekey</span>,</span> epay</span>,</span> enext</span> =</span> [</span>]</span>,</span> [</span>]</span>,</span> [</span>]</span></span>
    for</span> k</span>,</span> p</span> in</span> zip</span>(</span>keys</span>,</span> payloads</span>)</span>:</span></span>
        h</span> =</span> hash</span>(</span>k</span>)</span> &</span> mask</span></span>
        ekey</span>.</span>append</span>(</span>k</span>)</span>; </span>epay</span>.</span>append</span>(</span>p</span>)</span>; </span>enext</span>.</span>append</span>(</span>heads</span>[</span>h</span>]</span>)</span></span>
        heads</span>[</span>h</span>]</span> =</span> len</span>(</span>ekey</span>)</span> -</span> 1</span></span>
    return</span> {</span>"</span>heads</span>"</span>:</span> heads</span>,</span> "</span>ekey</span>"</span>:</span> ekey</span>,</span> "</span>epay</span>"</span>:</span> epay</span>,</span></span>
            "</span>enext</span>"</span>:</span> enext</span>,</span> "</span>mask</span>"</span>:</span> mask</span>}</span></span></code></pre>#</span> (probe, layout="open", in: dense, out: sel)</span></span>
def</span> probe_open</span>(</span>keys_in</span>,</span> ht</span>)</span>:</span></span>
    hits</span>,</span> pay</span> =</span> [</span>]</span>,</span> [</span>]</span></span>
    for</span> i</span>,</span> k</span> in</span> enumerate</span>(</span>keys_in</span>)</span>:</span></span>
        h</span> =</span> hash</span>(</span>k</span>)</span> &</span> ht</span>[</span>"</span>mask</span>"</span>]</span></span>
        while</span> ht</span>[</span>"</span>keys</span>"</span>]</span>[</span>h</span>]</span> !=</span> EMPTY</span>:</span>      #</span> scan the linear-probe run</span></span>
            if</span> ht</span>[</span>"</span>keys</span>"</span>]</span>[</span>h</span>]</span> ==</span> k</span>:</span></span>
                hits</span>.</span>append</span>(</span>i</span>)</span>; </span>pay</span>.</span>append</span>(</span>ht</span>[</span>"</span>pay</span>"</span>]</span>[</span>h</span>]</span>)</span></span>
                break</span></span>
            h</span> =</span> (</span>h</span> +</span> 1</span>)</span> &</span> ht</span>[</span>"</span>mask</span>"</span>]</span></span>
    return</span> hits</span>,</span> pay</span></span>
</span>
#</span> (probe, layout="chained", in: dense, out: sel)</span></span>
def</span> probe_chained</span>(</span>keys_in</span>,</span> ht</span>)</span>:</span></span>
    hits</span>,</span> pay</span> =</span> [</span>]</span>,</span> [</span>]</span></span>
    for</span> i</span>,</span> k</span> in</span> enumerate</span>(</span>keys_in</span>)</span>:</span></span>
        e</span> =</span> ht</span>[</span>"</span>heads</span>"</span>]</span>[</span>hash</span>(</span>k</span>)</span> &</span> ht</span>[</span>"</span>mask</span>"</span>]</span>]</span></span>
        while</span> e</span> !=</span> EMPTY</span>:</span>                  #</span> walk the chain</span></span>
            if</span> ht</span>[</span>"</span>ekey</span>"</span>]</span>[</span>e</span>]</span> ==</span> k</span>:</span></span>
                hits</span>.</span>append</span>(</span>i</span>)</span>; </span>pay</span>.</span>append</span>(</span>ht</span>[</span>"</span>epay</span>"</span>]</span>[</span>e</span>]</span>)</span></span>
                break</span></span>
            e</span> =</span> ht</span>[</span>"</span>enext</span>"</span>]</span>[</span>e</span>]</span></span>
    return</span> hits</span>,</span> pay</span></span></code></pre>lib</span>[</span>(</span>"</span>insert</span>"</span>,</span> "</span>open</span>"</span>)</span>]</span>                     =</span> build_open</span></span>
lib</span>[</span>(</span>"</span>insert</span>"</span>,</span> "</span>chained</span>"</span>)</span>]</span>                  =</span> build_chained</span></span>
lib</span>[</span>(</span>"</span>probe</span>"</span>,</span> "</span>open</span>"</span>,</span>    "</span>dense</span>"</span>,</span> "</span>sel</span>"</span>)</span>]</span>   =</span> probe_open</span></span>
lib</span>[</span>(</span>"</span>probe</span>"</span>,</span> "</span>chained</span>"</span>,</span> "</span>dense</span>"</span>,</span> "</span>sel</span>"</span>)</span>]</span>   =</span> probe_chained</span></span>
#</span> no ("probe", "robinhood", ...) entry exists</span></span></code></pre>sched</span>.</span>state_layout</span> =</span> {</span>"</span>ht</span>"</span>:</span> "</span>robinhood</span>"</span>}</span></span>
#</span> region_key for the probe region -> ("probe", "robinhood", "dense", "sel")</span></span>
#</span> not in lib -> IllegalSchedule on the precompiled tier, JIT otherwise</span></span></code></pre>Tier two: the JIT</h2>
When the lookup misses and the schedule isn’t pinned, plan</code> falls
through to the JIT, and in Python a JIT is pleasantly short^{5</a></sup> we can just generate
source, exec</code> it, return the function where jit_region</code> walks the ops in the region
and emits one line per suboperator a continue</code> guard per filter, a local per map and
an accumulation for the fold:</p>}
def</span> jit_region</span>(</span>ops</span>,</span> sched</span>,</span> start</span>,</span> end</span>,</span> needed_out</span>,</span> name</span>)</span>:</span></span>
    src</span> =</span> [</span>f</span>"</span>def </span>{</span>name</span>}</span>(chunk, state):</span>"</span>]</span></span>
    #</span> ... load the columns the region reads, pick dense or sel iteration,</span></span>
    #</span>     then one generated line per suboperator ...</span></span>
    exec</span>(</span>"</span>\n</span>"</span>.</span>join</span>(</span>src</span>)</span>,</span> ns</span> :=</span> {</span>"</span>Chunk</span>"</span>:</span> Chunk</span>}</span>)</span></span>
    return</span> ns</span>[</span>name</span>]</span></span></code></pre>
Unlike the library blocks, the generated code specializes the query into the source: column names
and constants are baked in, the way HyPer’s codegen bakes them into LLVM IR. Running hyper</code> with tracing
on prints what came out:</p>
region 0: ops[0:7] tier=jit</span></span>
| def region0(chunk, state):</span></span>
|     discount = chunk.cols['discount']</span></span>
|     extprice = chunk.cols['extprice']</span></span>
|     quantity = chunk.cols['quantity']</span></span>
|     shipdate = chunk.cols['shipdate']</span></span>
|     acc = state['acc']</span></span>
|     idx = chunk.sel if chunk.sel is not None else range(chunk.n)</span></span>
|     for i in idx:</span></span>
|         if not (shipdate[i] >= 8766): continue</span></span>
|         if not (shipdate[i] < 9131): continue</span></span>
|         if not (discount[i] >= 5): continue</span></span>
|         if not (discount[i] <= 7): continue</span></span>
|         if not (quantity[i] < 24): continue</span></span>
|         v_revenue = extprice[i] * discount[i]</span></span>
|         acc += v_revenue</span></span>
|     state['acc'] = acc</span></span></code></pre>
This is the fused loop we wanted to get back in the schedules section except we didn’t explicitely
write it instead we defined a schedule fuse_all(Q6)</code> and seven declarative sub-operators and the
code kind of falls out. If we change the schedule we end up with different code frof</code> produces two
smaller functions, one compacting survivors into lists at its boundary and one consuming them.</p>
One argument that keeps coming back in the HyPer and Umbra papers is that code generation is legitimately
about trading off compile times for execution times. In the case of Umbra, HyPer’s successor and more recenetly
Cedar this was addressed by tiering at the IR level by using two compilers.</p>
In our case because we can decide on schedules before running the query for the first time we an benefit from
vectorization and pre-compiled fragments without ever triggering a compile and instead only trigger the compile
for queries we judge as hot or for UDFs6</a></sup>.</p>

I want to interject for a moment and clarify something, a lot of database papers on query compilation don’t
seem to discuss the contextual trade-off with I/O i.e. when your data is available for processing and I think
this information makes the compile time discussion more interesting.</p>
</blockquote>

Another thing I wish to explore in the future is how much we can theoretically push down to process near data,
compilation is very attractive here because of how imaginative you can get e.g. processing near NIC or NVMe
which seems like a non-brainer for hyperscalers and hard to achieve in practice for your average database.</p>
</blockquote>
But note how we didn’t have to do any of that and instead we just leverage our scheduling to get tiers for free. Of
course there is a compile time to produce machine code out of C code but this cost can be modeled into the scheduling
model and you can decide when to keep using pre-compiled blocks versus when to generated fused kernels.</p>
Finally we define a reference interpreter to act as a correctness oracle and this comes from the same
model we’ve been cooking all along. The reference interpreter walks the pipeline a row at a time
and never looks at a schedule:</p>
def</span> reference</span>(</span>table</span>,</span> ops</span>)</span>:</span></span>
    acc</span> =</span> 0</span></span>
    for</span> i</span> in</span> range</span>(</span>nrows</span>(</span>table</span>)</span>)</span>:</span></span>
        row</span> =</span> {</span>c</span>:</span> col</span>[</span>i</span>]</span> for</span> c</span>,</span> col</span> in</span> table</span>.</span>items</span>(</span>)</span>}</span></span>
        for</span> o</span> in</span> ops</span>:</span></span>
            ...</span>   #</span> apply each suboperator to the row dict</span></span>
    return</span> acc</span></span></code></pre>
Every combination of (schedule, tier)</code> can be asserted against the reference interpreter and when
something goes wrong you can just flip one annotation at a time and look at the results.
If for example the selection-vector schedule on the precompiled tier returns a revenue slightly off the
reference rerun it with tier="jit"</code> and if it agrees then you know some precompiled block is at
fault. Back on the precompiled tier, flip the edges to dense one at a time and look at the map’s
dense vectors which isolates the precompiled sel-variant of the map.</p>
reference (row at a time, no schedule): revenue = 64967015</span></span>
</span>
hyper    revenue = 64967015 [ok]   tier = jit</span></span>
vec      revenue = 64967015 [ok]   tier = precompiled</span></span>
vec_sel  revenue = 64967015 [ok]   tier = precompiled</span></span>
rof      revenue = 64967015 [ok]   tier = jit</span></span></code></pre>If you already speak MLIR</h2>
Everything above can be restated in MLIR terms, and it is worth doing
because most of the infrastructure already exists.
LingoDB</a> showed
that query compilation fits MLIR’s progressive-lowering model, and
Jungmair’s declarative sub-operators are implemented as a dialect on
that stack. So the algorithm half of the split is already an MLIR
first citizen suboperators are ops, and the edges of the DAG are SSA
values so the schedule is just part of the type system.</p>
%rows      = subop.scan @lineitem</span></span>
             : !sched.stream<4 x i64, dense, 1024, buf></span></span>
%survivors = subop.filter %rows</span></span>
             : !sched.stream<2 x i64, sel, 1024, buf></span></span></code></pre>
The schedule-as-a-separate-program idea exists too. The transform
dialect</a>, Halide’s
descendant inside MLIR, expresses tiling and fusion decisions as
their own IR whose payload is another program, an edge schedule is a
transform script over the suboperator dialect: fuse these ops into
one region, materialize this value at that type, at this capacity.
The walkthrough’s Schedule</code> value and regions</code> function are a
pocket-sized version of exactly that.</p>
Tiering is also naturally a follow through of MLIR’s dialects picking a tier
means picking a lowering path in this case you lower a fused region through scf</code>
to llvm</code> and you get the JIT tier. Lower a region to a func.call</code> against a pre-existing
library of precompiled blocks and you get the fragment tier.</p>
What MLIR does not provide is the contents of that middle tier; someone still has to decide which fused blocks to
enumerate. LingoDB today lowers every pipeline through one flavor chosen by static rules; the edge-schedule proposal
allows you to turn that fixed path into a search space.</p>
Where the comparison is not correct</h2>
Halide has limits that query plans do not have. These limits let Halide
divide the algorithm from the schedule. Query plans break that division
in four ways:</p>

Stages hold state.</li>
Bounds change with the data.</li>
Domains have no order.</li>
Halide has no tier.</li>
</ul>
Stages hold state.</strong> A Halide stage is a pure function of its inputs
fold</code> is not a pure function, and a hash-table build is not a pure
function. The walkthrough has only one item of state: the executor keeps
one accumulator and Q14 shows the small version of the problem i.e. the hash
table is common state with a barrier and no edge annotation can put the
probe before the end of the build.</p>
Q1 is the TPC-H query that aggregates by group. Q1 shows the large
version of the problem. Its fold keeps one aggregate for each group.
Thus the edge into the fold does not carry values. The edge carries
updates to common state. The aggregation monoid controls which
reorderings and parallel splits are safe, and the schedule does not.</p>
You must show the correctness guarantee again for state. An effect
system on suboperators can do this. The effect system must show which
edges carry values and which edges carry state. Halide has related work
with the name rfactor</code>. rfactor</code> divides a reduction into partial
states, one for each thread. This is the best example to start from.</p>
Bounds change with the data.</strong> Halide knows its loop extents at compile
time. A query engine does not know the output cardinality of a filter
before it runs that filter. This cardinality controls which of the
subsequent schedules is the fastest. Thus you cannot select the correct
schedule before the query starts. Halide’s autoscheduler does not have
this problem.</p>
I think that you must not select one schedule. Do these steps instead,
as Excalibur does:</p>

Make a short list of schedules.</li>
Run each schedule in the list on one morsel.</li>
Keep the schedule that is the fastest.</li>
</ol>
Two schedules with one different annotation have one different region of
code. Thus you can identify the change that made the schedule faster. A
different post will tell you how to make the short list. It will also
tell you the cost of an incorrect selection.</p>
Domains have no order.</strong> Halide divides ordered grids into tiles. A
relation is a bag, and a bag has no order. But the important decisions
about blocks are tiles of a domain that has no order. Morsels and radix
partitions before a join are two examples. I think that a radix join is a
loop tile operation with a partition operation. I cannot show this yet,
and I want this result from the formalism.</p>
Halide has no tier.</strong> Halide always compiles. Thus Halide never selects
between a precompiled library and a JIT. Query engines must control the
time to the first tuple. Thus the tier is not an internal detail here.
Inkfuse, Excalibur, and jamii’s post are different mostly on this axis.</p>
Things I’m still not sure about</h2>
Whether the schedule language is complete. Morsel-driven parallelism
looks like a granularity annotation on the scan edge plus a rule
about which state is shared, but that rule is the part I can’t state
precisely yet, and the shared-state half is the same thing I couldn’t
express in CPS terms last time.</p>
Whether the coercion algebra stays closed once nulls and
variable-length data show up. Every representation in this post is a
vector of fixed-width values, which is the easy case. Validity
bitmaps and string payloads each add representation axes, and I don’t
know whether they add a manageable number of coercions or an
explosion of them.</p>
And whether this is a genuinely different chart or VOILA with types.
The honest version of the differentiation, as I see it today: VOILA
puts the flavor in the code a backend generates for the operators,
and Excalibur searches over generated code; the edge view puts the
paradigm in the intermediates between suboperators, and makes the
code a derived artifact of edge decisions plus a tier. The
walkthrough is small, but it is a proof of that last part: its
library blocks and its generated regions are both downstream of the
same seven dataclasses and a tuple of edge annotations. I think that
relocation is what makes legality-by-presence expressible at all. But I have talked myself into framings like this
before and been wrong about which part mattered.</p>
References</h2>

The walkthrough compiler from this post:
schedules.py</a>.</li>
Tim Gubner, Peter Boncz, Charting the Design Space of Query
Execution using VOILA</a>,
PVLDB 14.</li>
Tim Gubner, Peter Boncz, Excalibur: A Virtual Machine for Adaptive
Fine-grained JIT-Compiled Query Execution based on
VOILA</a>, PVLDB 16.</li>
Maximilian Bandle, Jana Giceva, Database Technology for the Masses:
Sub-Operators as First-Class
Entities</a>, PVLDB 14.</li>
Michael Jungmair, Jana Giceva, Declarative Sub-Operators for
Universal Data
Processing</a>,
PVLDB 16.</li>
Michael Jungmair, André Kohn, Jana Giceva, Designing an Open
Framework for Query Optimization and
Compilation</a>
(LingoDB), PVLDB 15.</li>
Prashanth Menon, Todd Mowry, Andrew Pavlo, Relaxed Operator Fusion
for In-Memory Databases</a>,
PVLDB 11.</li>
Jonathan Ragan-Kelley et al., Halide: A Language and Compiler for
Optimizing Parallelism, Locality, and Recomputation in Image
Processing Pipelines</a>,
PLDI 2013.</li>
Haoran Xu, Fredrik Kjolstad, Copy-and-Patch
Compilation</a>, OOPSLA 2021.</li>
Jamie Brandon, A missing tier for query
compilers</a>.</li>
Timo Kersten et al., Everything You Always Wanted to Know About
Compiled and Vectorized Queries But Were Afraid to
Ask</a>, PVLDB 11.</li>
The Inkfuse
paper</a>,
and the previous post</a> on it.</li>
</ul>



The raptor</code> implementation is not opensource yet because it’s
still not complete and it’s still just a tinkering project I
work on from time to time. ↩</a></p>
</li>

I am fairly sure with some extra effort one can describe this
as an algebra and work out the formal details. ↩</a></p>
</li>

The compiler guaranteeing that a schedule is legal is very different
from encoding it into the rewrite rule. A handwavy analogy is how Rust
makes memory safety a compiler checked property versus having it as a
runtime property e.g. reference counting. ↩</a></p>
</li>

The longer version of this argument, with the latency
arithmetic, is in the last post’s footnote on ROF and
prefetching; the canonical reference is Chen et al., Improving
Hash Join Performance Through Prefetching</em>. ↩</a></p>
</li>

Changing Python for C gives us the same brevity a good design approach for
generating C code can be seen in this post</a> ↩</a></p>
</li>

UDFs are “la bete noire”</em> of query execution and offering UDF support almost always means supporting
another compiler. BigQuery for example supports Javascript for UDFs and while its not documented anywhere
if I were to venture a guess I would bet some form of stripped down V8 is used or maybe even just V8 is
used for those. The nice thing about UDFs is that while they can be black boxes, statistically you know
you will execute them many times ~ O(rows) so compiling them straight away will be rewarding.
An intuitive way to think about it is that you dispatch the UDF to both TurboFan (fast compile times)
and Maglev (V8’s optimizing compiler) and whichever finishes first can be used to process available data.
This approach is only rewarding if you expect your I/O to be super fast e.g. if you dispatch I/O’s before
lowering the final plan as a sort of speculative warm up but if your I/O is over the network e.g. reading Parquet
over S3 then you can just compile with the last tier. ↩</a></p>
</li>
</ol>
</section>
Opinions on Software.

Execution Paradigms Are Schedules

Inkfuse, Continuations and Query Compilation

Opinions on Software.

Execution Paradigms Are Schedules

Physical Layouts as Schedule Decisions</h2> Another thing that needs to be encoded is the physical layout of hashmap buckets this only matters because our approach allows us to define an arbitrary layout for prob/build</code> hashmaps and any storage interface.</p>

Where the comparison is not correct</h2> Halide has limits that query plans do not have. These limits let Halide divide the algorithm from the schedule. Query plans break that division in four ways:</p> Stages hold state.</li> Bounds change with the data.</li> Domains have no order.</li>

Inkfuse, Continuations and Query Compilation
