Model-Serving
All articles
Zero Trust for AI Model Serving
Applying zero trust architecture to AI systems: securing inference endpoints, model artifact access, training …vLLM - High-Performance LLM Serving Engine
vLLM is an open-source library for high-throughput, low-latency serving of large language models using …Ollama - Local LLM Inference Engine
Ollama is an open-source tool for running large language models locally on personal hardware with a simple …Kubeflow - Machine Learning Platform for Kubernetes
Kubeflow is an open-source machine learning platform that makes deploying, scaling, and managing ML workflows …gRPC vs REST for AI/ML Microservices
Comparing gRPC and REST for serving AI models in microservice architectures, covering performance, developer …Inference - Running AI Models in Production
What inference means in AI context, the key operational parameters that matter (latency, throughput, cost), …
Open source projects