In November, work continued on revamping Array evaluation to be fully lazy. This happens by converting their execution to an Operator model that evaluates into Vectors (fully decompressed, zero-copy to Arrow representation). As a reminder, this work enables many more optimizations, and also provides unified abstractions for evaluating on different processor types (CPUs & GPUs). We anticipate shipping lazy Array evaluation by the end of the year.
Around the Ecosystem
- Spice AI launched the Spice Cayenne Data Accelerator based on Vortex
- Polar Signals replaced Parquet with Vortex for their profiling database
Given how well Vortex’s design matched our needs, we tried it out and got a 70% average performance improvement on all our queries.
- Vortex is now a DuckDB core extension only and was removed from the list of community releases. To install:
INSTALL vortex;
Discussions
Feel free to contribute to the following recent discussions:
- Release process for breaking API changes #5624
- Benchmark restructuring #5674
- To upvote priority on Vortex integrations, please comment on #5525.
Acknowledgments
We want to thank to anyone who has tried Vortex, provided feedback, asked question and filed issues. Special congratulations to Alfonso Subiotto Marqués on joining the Technical Steering Committee as a committer.
The following contributed to the November releases.
Joe Isaacs
Connor Tsui
Nicholas Gates
Robert Kruszewski
Andrew Duffy
Alexander Droste
Dan King
Onur Satici
Adam Gutglick
Alfonso Subiotto Marqués
Will Manning
Dmitrii Blaginin
Frederic Branczyk
James Xu
Nick Cameron
