Today, Community Labs is launching Cascadia, a distributed AI inference runtime for Intel hardware. Developed in collaboration with Intel, Cascadia allows organizations to turn their existing Intel-powered fleets into a private system for running large language models.

Organizations increasingly rely on AI tools for their workflows, yet much of their existing compute sits idle in favor of using cloud APIs. Cascadia is designed to put that hardware to use through intelligent model sharding and routing, without third-party token costs and keeping sensitive data onsite.

Cascadia supports three different ways to serve models locally: one model on a single device, replicate a model across devices, or splitting a larger and more powerful model across devices.

“We built Cascadia around a simple premise: people should not have to choose between cloud AI models or purchasing dedicated hardware,” said Tate Berenbaum, founder of Community Labs. “Working with Intel, we have shown that a third option is possible: running capable models on the hardware that you already own.”

Tate Berenbaum has spent the last six years working on distributed systems, with a focus on enabling sophisticated workloads to run on commodity hardware.

Cascadia splits large AI models into shards and uses Intel’s OpenVINO toolkit to run optimized inference across CPUs, integrated GPUs, and NPUs. It also supports a range of Intel’s Arc Pro discrete graphics cards.

Initial usage has shown promising results with two research papers covering the work to be released soon.

Cascadia is designed for organizations and individuals who want greater control over how their AI models run. For example, teams with data-residency or privacy requirements, or enterprises looking to reduce their usage-based token costs.

“Intel is focused on engineering the next wave of computing, where AI workloads are distributed, efficient, and closer to where data is generated,” said Dennis Luo, Senior Director and General Manager, Worldwide Developer Relations & Innovation, Intel. “We’re excited to work with Cascadia to enable AI PCs to run models previously impossible and help bring efficient distributed AI to more environments.”

While many on-prem solutions require dedicated hardware and software for AI inference, Cascadia is designed so that every computer on a network can contribute their idle compute. Each PC runs a single program which handles discovery, routing, and execution of requests across the network.

Cascadia’s runtime is open source and OpenAI API-compatible so it can slot into many existing AI workflows. For enterprises, Cascadia offers a richer feature set optimized for managing fleets of devices and users across networks, end-to-end deployment, and operational support.

Visit cascadia.to to learn more.

About Cascadia

Cascadia turns commodity Intel hardware into a private AI inference cluster. Run powerful local AI models across Intel CPUs, iGPUs, NPUs, and discrete GPUs. No cloud subscription required.

Media gallery

About The Author
Business Wire