AI you can trust with your
Kalosm is an open source framework for private language, audio, and image models in Rust
microphone_rag.rs
app.rs
let model = WhisperBuilder::default() .with_source(WhisperSource::MediumEn) .build()?; let db = Surreal::new::<RocksDb>("./db/temp.db").await.unwrap(); db.use_ns("test").use_db("test").await.unwrap(); let document_table = Arc::new(db.document_table_builder("documents").at("./db/embeddings.db").build().unwrap()); let document_table_clone = document_table.clone(); std::thread::spawn(move || tokio::runtime::Runtime::new().unwrap().block_on(async move { loop { let input = MicInput::default().record_until(Instant::now() + Duration::from_secs(30)).await.unwrap(); if let Ok(mut transcribed) = model.transcribe(input) { while let Some(transcribed) = transcribed.next().await { document_table_clone.insert(transcribed.text().into_document().await.unwrap()).await.unwrap(); } } } })); let mut model = Llama::new_chat(); let mut chat = Chat::builder(&mut model).with_system_prompt("The assistant help answer questions based on the context given by the user. The model knows that the information the user gives it is always true.").build(); let context = document_table.select_nearest(&prompt_input("\n> ").unwrap(), 5).await?.into_iter().map(ToString::to_string).collect::<Vec<_>>().join("\n"); let output_stream = chat.add_message(format!("Here is the relevant context:\n{context}\nGiven that context, answer the following question:\n{user_question}")).await
Powerful Private AI
Everything you need to build with local AI
Kalosm is a powerful, private AI platform for building and deploying AI models locally. It's designed to be easy to use, controllable, and private.
- AI Models
- Kalosm supports over 35 models across 5 different model types. Models can be easily swapped out and updated depending on your hardware requirements and use case.
- Local Processing
- Kalosm models run locally which means no expensive cloud bills, connection issues, or privacy concerns. User data never needs to leave the device.
- Data Integrations
- Kalosm understands 10 different data formats making it easy to integrate with your text, audio, or image data.
- Controllable Output
- Kalosm is built with controls to tailor the output of your models. This means you can adjust the output to fit your specific use case.