[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"$fk3Q0iz8AVg-GPsQN1fkPyuP1Rt4VSTY1PJxVZpTMexI":3},{"item":4,"related":43},{"id":5,"type":6,"title":7,"slug":8,"summary":9,"body":10,"coverUrl":11,"productScreenshots":12,"productLinks":13,"authorName":14,"authorUrl":15,"authorSubject":16,"category":17,"tags":22,"sourceLabel":35,"sourceName":35,"sourceUrl":35,"status":36,"seoTitle":35,"seoDescription":35,"canonicalUrl":35,"isFeatured":37,"sno":38,"sortOrder":39,"publishedAt":40,"updatedAt":41,"createdAt":42},"0e2211d6-a9cc-4151-aec3-ea60ef2575f2","article","本地大模型部署：用 Ollama 与 llama.cpp 把模型搬进你自己的机器","local-llm-deployment-ollama-llama-cpp","数据敏感、要离线、想省 API 账单？本地部署值得了解。","把大模型搬到你自己的电脑、内网服务器甚至笔记本上跑，不依赖任何云服务——这件事在 2026 年已经相当成熟。无论是数据敏感、要离线、还是想省 API 账单，本地大模型部署都值得每个开发者了解。Ollama 和 llama.cpp 是这条路上最顺手的两件工具。\n\n## 为什么要在本地跑\n\n云端 API 方便，但有三类痛点它躲不开：数据要出网（合规敏感场景直接否决）、每次调用都计费、断网就歇菜。本地部署把模型权重放在你自己的机器上，请求不出内网、零边际成本、永远在线。代价是你要自己搞定硬件和推理环境。\n\n## 两件核心工具\n\n**Ollama**：把「下载模型、起服务、调接口」封装成几条命令，对开发者最友好，自带兼容 OpenAI 的接口。\n**llama.cpp**：用 C++ 实现、支持量化与多后端（CPU\u002FGPU\u002FMetal），是把模型塞进低配机器的底层引擎，很多上层工具（包括 Ollama）都站在它肩上。\n\n```mermaid\nflowchart LR\n    A[模型权重文件] --> B[llama.cpp 推理引擎]\n    B --> C[Ollama 封装服务]\n    C --> D[你的应用 走 OpenAI 兼容接口]\n```\n\n## 一个最小可运行的例子\n\n用 Ollama 跑起一个模型并调用，比想象中简单：\n\n```bash\nollama pull qwen2.5:7b     # 拉取一个 70 亿参数模型\nollama run qwen2.5:7b      # 命令行直接对话\n```\n\n在 Python 里，它可以像调云端一样用：\n\n```python\nfrom ollama import chat\nresp = chat(model=\"qwen2.5:7b\", messages=[\n    {\"role\": \"user\", \"content\": \"用一句话解释什么是向量数据库\"}\n])\nprint(resp[\"message\"][\"content\"])\n```\n\n## 取舍与边界\n\n- **硬件是硬门槛**：7B 模型量化后约 4–5 GB 显存，能跑；70B 级别需要大显存或多卡，笔记本基本没戏。\n- **质量有差距**：本地小模型（7B\u002F14B）在复杂推理上仍明显弱于云端旗舰模型，适合内部工具、草稿、分类等场景。\n- **量化换速度**：用 llama.cpp 的 INT4 量化能在 CPU 上跑起来，但精度会降，关键任务先评测。\n- **并发能力弱**：本地单机吞吐远不及云厂商集群，不适合高并发公网服务。\n\n## Tips\n\n- 想试水，先 `ollama pull` 一个 7B 模型，五分钟跑通对话。\n- 应用层尽量走 OpenAI 兼容接口，本地\u002F云端切换只改 base_url。\n- 数据敏感或要离线，本地部署是合规最优解。\n- 真要上生产高并发，把本地模型定位为「内网辅助」，重活仍交给云端旗舰。\n- 选模型时先想清楚硬件：显存不够就上量化版，别硬刚全精度。\n","https:\u002F\u002Foxqtewbrpuiouqqjrvdv.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fpublic-media\u002F2026-07-20\u002Fbf471546-ff4c-4fee-a01c-8a41157e5a8c.jpg",[],[],"Foundit AI","https:\u002F\u002Ffoundit.cn","f39339b1-aaa6-4e86-b0c2-a6e6a21113b5",{"id":18,"name":19,"slug":20,"description":21},"d6750616-07d9-4350-8485-1834c77be3d2","指南","guide","指导建议，仅供参考",[23,27,31],{"id":24,"name":25,"slug":26},"7c76bfc2-f80f-4ee0-a95d-27bd8708b434","技术","slug",{"id":28,"name":29,"slug":30},"a202d639-99a6-488a-a712-4d4c6ffd7e15","开发","dev",{"id":32,"name":33,"slug":34},"88d2bc27-0e0f-468a-b907-2991cb97b87b","人工智能","ai",null,"published",false,76,0,"2026-07-19T00:00:00.000Z","2026-07-20T01:22:41.329Z","2026-07-20T01:11:41.120Z",[44,54,62],{"id":45,"type":6,"title":46,"slug":47,"summary":48,"coverUrl":49,"authorName":50,"sno":51,"publishedAt":52,"createdAt":53},"5ef7ce44-0678-40b4-95e0-af4fe8a6b0a7","提示词也能缓存：固定前缀为什么能省钱提速？","prompt-caching-prefix-kv-cache","Prompt caching 缓存的不是旧答案，而是模型处理重复提示词前缀时产生的中间状态。本文解释它与语义缓存的区别、为什么顺序会影响命中、如何整理 Agent 上下文，以及多租户场景中的隔离风险。","\u002Fuploads\u002F2026-09-08\u002F182822f9-05a3-4d2b-8766-bc5daf93effd.jpg","Foundit",42,"2026-09-08T00:00:00.000Z","2026-09-08T03:19:26.522Z",{"id":55,"type":6,"title":56,"slug":57,"summary":58,"coverUrl":59,"authorName":50,"sno":60,"publishedAt":52,"createdAt":61},"7f7b281e-b9d6-406f-8d79-9dfb33145f5e","WebTransport：为什么实时 AI 应用不一定应该使用 WebSocket？","webtransport-realtime-ai-apps","WebSocket 适合通用双向消息，但复杂实时 AI 应用还可能需要可靠流、双向流和可以丢弃的临时数据。本文解释 WebTransport 的 session、stream 和 datagram，比较它与 SSE、WebSocket、WebRTC 的边界，并讨论鉴权与部署。","\u002Fuploads\u002F2026-09-08\u002F2247cab5-87b7-4b92-8871-e6adac14dc47.jpg",44,"2026-09-08T03:19:19.069Z",{"id":63,"type":6,"title":64,"slug":65,"summary":66,"coverUrl":67,"authorName":50,"sno":68,"publishedAt":69,"createdAt":70},"0c00fd8d-379a-4c58-a542-460bb3575b72","MCP Apps：让 AI 对话里的工具带上交互界面","mcp-apps-interactive-tool-ui","MCP Apps 通过 ui:\u002F\u002F 资源、工具元数据和沙箱 iframe，让 MCP 服务器可以向宿主提供交互式 HTML 界面。本文解释工具与 UI 的关联、权限边界、文本回退和安全模型。","\u002Fuploads\u002F2026-09-12\u002F9e795ab0-aa98-480f-ad6d-a2013530b302.jpg",48,"2026-09-12T00:00:00.000Z","2026-09-12T03:56:44.551Z"]