{"id":172,"date":"2026-04-19T23:32:06","date_gmt":"2026-04-19T15:32:06","guid":{"rendered":"https:\/\/www.chaohao.net.cn\/?p=172"},"modified":"2026-05-28T22:51:30","modified_gmt":"2026-05-28T14:51:30","slug":"llama-cpp%e9%83%a8%e7%bd%b2%e5%8f%8avs-code%e6%8f%92%e4%bb%b6continue%e8%bf%9e%e6%8e%a5","status":"publish","type":"post","link":"https:\/\/www.chaohao.net.cn\/?p=172","title":{"rendered":"llama.cpp\u90e8\u7f72\u53caVS Code\u63d2\u4ef6Continue\u8fde\u63a5"},"content":{"rendered":"\n

\u5f00\u7bc7\u5462\uff0c\u5148\u7ed9\u81ea\u5df1\u53e0\u4e2a\u7532\uff0c\u535a\u4e3b\u4e5f\u662f\u82b1\u4e86\u4e9b\u65f6\u95f4\u624d\u5b8c\u6210\uff0c\u6240\u4ee5\u522b\u5acc\u535a\u4e3b\u5570\u55e6\uff0c\u6211\u8ba4\u4e3a\u5570\u55e6\u624d\u662f\u4e00\u4e2a\u4eba\u5355\u72ec\u5b8c\u6210\u8be5\u90e8\u7f72\u7684\u72ec\u6709\u7559\u767d\u3002\u5982\u6709\u4e0d\u9002\uff0c\u8bf7\u7acb\u5373\u79bb\u5f00\uff0c\u4ee5\u9632\u751f\u6c14\u3002\u535a\u4e3b\u4e00\u65b9\u9762\u4e5f\u662f\u4e3a\u81ea\u5df1\u8c0b\u4e2a\u540e\u8def\uff0c\u53e6\u4e00\u65b9\u9762\u5e0c\u671b\u80fd\u591f\u5e2e\u52a9\u5230\u60f3\u8981\u5f97\u5230\u5e2e\u52a9\u7684\u4eba\u3002<\/p>\n\n\n\n

\u4e00\uff1a<\/p>\n\n\n\n

\u9996\u5148\u5462\uff0c\u8981\u786e\u8ba4\u9002\u5408\u60a8\u7684\u6a21\u578b\u5927\u5c0f\uff0c\u4f8b\u5982\u6211\u7684\u7b14\u8bb0\u672c\u914d\u7f6e\uff1a<\/p>\n\n\n\n

\"\"<\/div><\/figure>\n\n\n\n

,\u9002\u54087B-9B\uff0c\u6a21\u578b\u53ef\u5b8c\u5168\u52a0\u8f7d\u81f3\u663e\u5b58\uff0c\u54cd\u5e94\u5982\u98de\uff0c\u4f53\u9a8c\u6d41\u7545\uff0c\u7136\u540e\u5462\u53bb\u5b98\u65b9\u6587\u6863\u5185\u9605\u89c8\u76f8\u5173\u6587\u4ef6\u548c\u8981\u6c42\u5185\u5bb9\uff1a\uff08 https:\/\/github.com\/ggml-org\/llama.cpp<\/a> \uff09\uff0c\u8981\u662f\u5acc\u9ebb\u70e6\u5450\uff0c\u6084\u6084\u544a\u8bc9\u60a8\uff0c\u8bbf\u95ee\uff08https:\/\/zread.ai\/ggml-org\/llama.cpp<\/a>\uff09\uff0c\u53bb\u62c6\u89e3\u6587\u6863\u5185\u5bb9\u3002<\/p>\n\n\n\n

\"\"<\/div><\/figure>\n\n\n\n

\u4e8c\uff1a<\/p>\n\n\n\n

\u52a8\u52a8\u53d1\u8d22\u7684\u5c0f\u624b\uff0c\u563f\uff0c\u70b9\u51fb\u56fe\u7247\u4e2d\u7684\u4f4d\u7f6e\uff1a<\/p>\n\n\n\n

\"\"<\/div><\/figure>\n\n\n\n

\uff0c\u60a8\u5c31\u5f97\u5230\u4e86\u6587\u4ef6\u7684\u4e0a\u4f20\u5386\u53f2\u8bb0\u5f55\u4e86\u3002<\/p>\n\n\n\n

\"\"<\/div><\/figure>\n\n\n\n

,\uff08\u5bf9\u4e8ellama.cpp\uff0c\u5e94\u8be5\u4f18\u5148\u9009\u62e9\u5e26\u6709\u201cGGUF\u201d\u5b57\u6837\u7684\u91cf\u5316\u6a21\u578b\u6587\u4ef6\uff0c\u5176\u4e2dGGUF \u662f llama \u9879\u76ee\u4e13\u95e8\u5b9a\u4e49\u7684\u4e00\u79cd\u6a21\u578b\u6587\u4ef6\u683c\u5f0f\uff0c\u5b83\u628a\u6a21\u578b\u7ed3\u6784\u548c\u91cf\u5316\u540e\u7684\u6570\u636e\u6253\u5305\u5728\u4e00\u8d77\uff0c\u80fd\u8ba9 llama\u76f4\u63a5\u3001\u9ad8\u6548\u5730\u52a0\u8f7d\u8fd0\u884c\u3002\uff09\u597d\uff0c\u4e0b\u8f7d\u597d\u7248\u672c\uff0c\u5efa\u8bae\u662f\u5728\u8fc5\u96f7\u4e0a\u4e0b\u8f7d\uff0c\u56e0\u4e3a\u8fd9\u6837\u76f8\u5bf9\u6765\u8bf4\u5feb\u4e00\u70b9\u3002\uff08ps:\u535a\u4e3b\u4f7f\u7528\u7684\u662f\u6821\u56ed\u7f51\uff0c\u6240\u4ee5\u4e0b\u8f7d\u6162\uff0c\u624d\u91c7\u7528\u4e86\u8fc5\u96f7\u4e0b\u8f7d\uff0c\u4e00\u65e6\u8fc5\u96f7\u6ca1\u6709\u5468\u56f4\u8d44\u6e90\u7684\u8bdd\uff0c\u5c31\u65e0\u6cd5\u4e0b\u8f7d\uff0c\u6240\u4ee5\u4e00\u5207\u770b\u73af\u5883\uff09<\/p>\n\n\n\n

\u4e09\uff1a<\/p>\n\n\n\n

\u4e0b\u8f7d\u540e\uff0c\u518d\u6765\u4e0b\u8f7d\u672c\u5730\u7684\u6a21\u578b\uff1a\uff08https:\/\/hf-mirror.com\/models<\/a>\uff09\uff0c\u6311\u9009\u4f60\u6240\u9002\u914d\u7684\u5927\u6a21\u578b\u3002<\/p>\n\n\n\n

\"\"<\/div><\/figure>\n\n\n\n

,\u5728\u6587\u4ef6\u5217\u8868\u4e2d\u627e\u5230 Qwen3.5-9B-UD-Q4_K_XL.gguf<\/code>\uff0c\u70b9\u51fb\u4e0b\u8f7d\u3002\uff08\u56fe\u7247\u4f5c\u4e3a\u53c2\u8003\uff09\u3002<\/p>\n\n\n\n

\u56db\uff1a<\/p>\n\n\n\n

\u60a8\u6253\u5f00\u7684\u8fd9\u4e2a\u6587\u4ef6\u5939 (cudert-llama-bin-win-cuda-12.4-x64) \u53ea\u6709\u4e09\u4e2a .dll \u6587\u4ef6\uff0c\u5b83\u4eec\u662f\u8fde\u63a5 llama\u548c\u4f60\u7684 NVIDIA \u663e\u5361\u8fdb\u884c\u52a0\u901f\u7684\u6865\u6881\u3002\uff08\u8fd9\u91cc\u535a\u4e3b\u4e5f\u662f\u72af\u4e86\u96be\uff0c\u6ca1\u6709.exe\u8fd9\u600e\u4e48\u6253\u5f00\u5462\uff0c\u4f46\u662f\u8fd9\u4e09\u4e2a\u6587\u4ef6\u53ef\u4e8b\u5173\u91cd\u8981\uff01\uff09\uff0c\u76f8\u5e94\u7684\u6587\u4ef6\u5728\u8fd9\uff1a(https:\/\/github.com\/ggml-org\/llama.cpp\/releases\/download\/b8840\/llama-b8840-bin-win-cuda-12.4-x64.zip<\/a>)\u3002\u4e0b\u8f7d\u5b89\u88c5\u540e\u5efa\u8bae\u5c06\u5730\u5740\u653e\u5728\u719f\u6089\u7684\u4f4d\u7f6e\uff0c\u4ee5\u514d\u5fd8\u8bb0\u5730\u5740\u3002<\/p>\n\n\n\n

\u4e94\uff1a\u627e\u5bfb\u5230\u8be5\u5730\u5740\u540e\uff0c\u5148\u5c06\u4e09\u4e2a.dll\u653e\u5165\u5728llama.cpp\u6587\u4ef6\u5185\uff0c\u76f4\u63a5\u526a\u5207\u7c98\u8d34\u5c31\u884c\uff0c\u7136\u540e\u6253\u5f00\u8be5\u6587\u4ef6\u7684cmd\u7a97\u53e3\uff0c\u5982\u679c\u8fd9\u4e2a\u4e0d\u77e5\u9053\u600e\u4e48\u64cd\u4f5c\u7684\u8bdd\uff0c\u4f60\u8be5\u597d\u597d\u7684\u5b66\u4e60\u4e86\uff01<\/p>\n\n\n\n

\"\"<\/div><\/figure>\n\n\n\n

\uff0c\u70b9\u51fb\u7bad\u5934\u6240\u6307\u4f4d\u7f6e\uff0c\u8f93\u5165cmd\uff0c<\/p>\n\n\n\n

 .\\llama-cli.exe -m \"E:\\Models\\Qwen3.5-9B-UD-Q4_K_XL.gguf\" -ngl 999 --color auto<\/pre>\n\n\n\n
\uff08\u8fd9\u91cc\u7684\u6587\u4ef6\u4f4d\u7f6e\u81ea\u5df1\u8bbe\u7f6e\uff0c\u522b\u8001\u5b9e\u5df4\u4ea4\u7684\u5168\u6309\u7167\u6211\u7684\u8def\u5f84\u6765\u4e86\u3002\u8fd8\u6709\u8def\u5f84\u4e00\u5b9a\u8981\u5168\u82f1\u6587\u8def\u5f84\uff0c\u4e0d\u7136\u4f1a\u7206\u9519\uff01\uff1a
(eg:<\/p>\n\n\n\n
E:\\llama-b8840-bin-win-cuda-12.4-x64>.\\llama-cli.exe -m “D:\\\u8fc5\u96f7\u4e0b\u8f7d\\Qwen3.5-9B-UD-Q4_K_XL.gguf” -ngl 999 –color auto
load_backend: loaded RPC backend from E:\\llama-b8840-bin-win-cuda-12.4-x64\\ggml-rpc.dll
load_backend: loaded CPU backend from E:\\llama-b8840-bin-win-cuda-12.4-x64\\ggml-cpu-alderlake.dll<\/p>\n\n\n\n
Loading model… |gguf_init_from_file: failed to open GGUF file ‘D:\\\u0478\ufffd\ufffd\ufffd\ufffd\ufffd\ufffd\\Qwen3.5-9B-UD-Q4_K_XL.gguf’ (No such file or directory)
llama_model_load: error loading model: llama_model_loader: failed to load model from D:\\\u0478\ufffd\ufffd\ufffd\ufffd\ufffd\ufffd\\Qwen3.5-9B-UD-Q4_K_XL.gguf
llama_model_load_from_file_impl: failed to load model
llama_params_fit: encountered an error while trying to fit params to free device memory: failed to load model llama_model_load: error loading model: llama_model_loader: failed to load model from D:\\\u0478\ufffd\ufffd\ufffd\ufffd\ufffd\ufffd\\Qwen3.5-9B-UD-Q4_K_XL.gguf
llama_model_load_from_file_impl: failed to load model
common_init_from_params: failed to load model ‘D:\\\u0478\ufffd\ufffd\ufffd\ufffd\ufffd\ufffd\\Qwen3.5-9B-UD-Q4_K_XL.gguf’
srv load_model: failed to load model, ‘D:\\\u0478\ufffd\ufffd\ufffd\ufffd\ufffd\ufffd\\Qwen3.5-9B-UD-Q4_K_XL.gguf’<\/p>\n\n\n\n
Failed to load the model\uff09 \uff09<\/p>\n\n\n\n
\u516d\uff1a
\u5982\u679c\u51fa\u73b0\u4ee5\u4e0a\u5185\u5bb9\uff0c\u8bf4\u660e\u60a8\u5df2\u7ecf\u5b89\u88c5\u6210\u529f\u4e86\uff0c\u5e76\u4e14\u53ef\u4ee5\u4f7f\u7528\u4e86\u3002<\/p>\n\n\n\n
\"\"<\/div><\/figure>\n\n\n\n
\u4e03\uff1a<\/p>\n\n\n\n
\u8fd9\u91cc\u6765\u89e3\u91ca\u547d\u4ee4\u884c\u7a97\u53e3(CLI)\u4e2d\u7684\u542b\u4e49\uff1a<\/p>\n\n\n\n
 .\\llama-server.exe -m \"E:\\Models\\Qwen3.5-9B-UD-Q4_K_XL.gguf\" -ngl 999 -c 4096 -fa on -ctk q8_0 -ctv q8_0 -t 4 --host 0.0.0.0 --port 8080\n\n<\/pre>\n\n\n\n
-m : \u6a21\u578b\u8def\u5f84 | \u6211\u7684\u503c\uff1a\"E:\\Models\\Qwen3.5-9B-UD-Q4_K_XL.gguf\"\n-ngl \uff1a\u52a0\u8f7d\u5230GPU\u7684\u5c42\u6570 | 999 \uff1a(\u5c06\u6240\u6709\u7684\u5c42\u6570\u52a0\u8f7d\u5230RTX 4060\u663e\u5b58\uff09\n-c : \u4e0a\u4e0b\u6587\u957f\u5ea6\uff08\u6700\u5927\u652f\u6301\u7684token\u6570\uff09| 4096 \uff08\u53ef\u5904\u7406\u7ea63000\u6c49\u5b57\u7684\u5bf9\u8bdd\u5386\u53f2\uff09  ps:\u8fd9\u91cc\u5e94\u8be5\u8bbe\u518d\u5927\u4e00\u70b9\uff0c\u4e0d\u7136\u4e0d\u591f\u7528\u4f1a\u5bfc\u81f4\u5bf9\u8bdd\u5361\u58f3\n-fa on \uff1a\u542f\u7528Flash Attention,\u8282\u7ea6\u663e\u5b58\u5e76\u589e\u52a0\u63a8\u7406 | on\n-ctk q8_0 \uff1a \u5c06KV\u7f13\u5b58\u4e2d\u7684Key\u77e9\u9635\u91cf\u5316\u4e3a8-bit,\u51cf\u5c11\u663e\u5b58\u5360\u7528 | q8_0\n-ctv q8_0 : \u5c06KV\u7f13\u5b58\u4e2d\u7684Value\u77e9\u9635\u91cf\u5316\u4e3a8-bit | q8_0\n-t \uff1a CPU\u7ebf\u7a0b\u6570 | 4 \uff08\u907f\u514d\u7ebf\u7a0b\u8fc7\u591a\u5bfc\u81f4\u8c03\u5ea6\u62e5\u5835\uff09\n--host : \u670d\u52a1\u5668\u76d1\u542c\u7684IP\u5730\u5740 | 0.0.0.0 (\u5141\u8bb8\u4efb\u4f55\u8bbe\u5907\u8bbf\u95ee\u672c\u673aAPI\uff09\n--port : \u670d\u52a1\u5668\u76d1\u542c\u7684\u7aef\u53e3\u53f7 | 8080 \uff08\u53ef\u901a\u8fc7http:\/\/localhost:8080\u8bbf\u95ee\uff09<\/pre>\n\n\n\n
–host 0.0.0.0 \u548c –port 8080 \u7684\u4f5c\u7528\uff1a<\/p>\n\n\n\n
    \n
  • –host 0.0.0.0 \uff1a\u8ba9 API \u670d\u52a1\u76d1\u542c\u672c\u673a\u6240\u6709\u7f51\u7edc\u63a5\u53e3<\/strong>\u3002\u8fd9\u610f\u5473\u7740\uff1a\n
      \n
    • \u672c\u673a\u7a0b\u5e8f\u53ef\u4ee5\u901a\u8fc7 127.0.0.1.8080\u6216 localhost:8080\u8bbf\u95ee\u3002<\/li>\n\n\n\n
    • \u540c\u4e00\u5c40\u57df\u7f51\u5185\u7684\u5176\u4ed6\u8bbe\u5907\uff08\u5982\u624b\u673a\u3001\u5e73\u677f\u3001\u53e6\u4e00\u53f0\u7535\u8111\uff09\u4e5f\u53ef\u4ee5\u8f93\u5165\u4f60\u7535\u8111\u7684\u5c40\u57df\u7f51 IP\uff08\u4f8b\u5982 192.168.1.5:8080<\/code>\uff09\u6765\u8c03\u7528\u6a21\u578b\uff08\u9700\u5173\u95ed\u9632\u706b\u5899\u6216\u653e\u884c\u7aef\u53e3\uff09\u3002–<\/li>\n<\/ul>\n<\/li>\n\n\n\n
    • –port 8080 \uff1a\u6307\u5b9a API \u670d\u52a1\u4f7f\u7528\u7684\u7aef\u53e3\u53f7\u3002\u4f60\u53ef\u4ee5\u6539\u6210\u4efb\u610f\u672a\u5360\u7528\u7684\u7aef\u53e3\uff08\u598211434\u30015000 \u7b49\uff09\u3002<\/li>\n<\/ul>\n\n\n\n
      \u516b\uff1a<\/p>\n\n\n\n
      \u5982\u679c\u60a8\u9700\u8981\u548cVS Code\u914d\u5408\u4f7f\u7528\u7684\u8bdd\uff0c\u8fd8\u8bf7\u60a8\u518d\u5f80\u4e0b\u77a7\uff1a<\/p>\n\n\n\n
      \u5b89\u88c5 Continue \u63d2\u4ef6\uff1a<\/p>\n\n\n\n
      \"\"<\/div><\/figure>\n\n\n\n
      \u4e4b\u540e\u5462\u5bfb\u627e\u8be5\u6587\u4ef6\u7684\u4f4d\u7f6e\uff1a\u4e00\u822c\u5728 C:\\Users\\admin\\.continue \u5185\uff0c\u76ee\u524d\u6211\u4f7f\u7528\u7684\u662fjson\u6587\u4ef6\uff0c\u636e\u8bf4\u8fd9\u4e2ajson\u6587\u4ef6\u5c06\u4f1a\u88ab\u4ee3\u66ff\u3002<\/p>\n\n\n\n
      {<\/p>\n\n\n\n
        “$schema”: “https:\/\/schema.continue.dev\/config.json”,<\/p>\n\n\n\n
        “name”: “continue-config”,<\/p>\n\n\n\n
        “version”: “1”,<\/p>\n\n\n\n
        “models”: [<\/p>\n\n\n\n
          {<\/p>\n\n\n\n
            “title”: “Qwen3.5-9B Local”,<\/p>\n\n\n\n
            “provider”: “openai”,<\/p>\n\n\n\n
            “model”: “AUTODETECT”,<\/p>\n\n\n\n
            “apiBase”: “http:\/\/localhost:8080\/v1”,<\/p>\n\n\n\n
            “apiKey”: “EMPTY”,<\/p>\n\n\n\n
            “roles”: [“chat”, “edit”, “apply”],<\/p>\n\n\n\n
            “noProxy”: true,<\/p>\n\n\n\n
            “timeout”: 60000<\/p>\n\n\n\n
          }<\/p>\n\n\n\n
        ]<\/p>\n\n\n\n
      }<\/p>\n\n\n\n
      \u4fee\u6539\u540e\u6587\u4ef6\u4fdd\u5b58\uff0c\u7136\u540e\u91cd\u65b0\u6253\u5f00VS Code \u5373\u53ef
      \u56de\u987e\u6574\u4e2a\u8fc7\u7a0b\uff0c\u60a8\u4ece\u4e00\u4e2a\u5bf9 llama.cpp \u5b8c\u5168\u964c\u751f\u7684\u8d77\u70b9\uff0c\u4e00\u6b65\u6b65\u89e3\u51b3\u4e86\u6587\u4ef6\u4e0b\u8f7d\u3001CUDA \u73af\u5883\u914d\u7f6e\u3001\u547d\u4ee4\u884c\u53c2\u6570\u4f18\u5316\u3001API \u670d\u52a1\u642d\u5efa\uff0c\u6700\u540e\u6210\u529f\u96c6\u6210\u5230 IDE\u3002\u8fd9\u79cd\u4ece\u5e95\u5c42\u7f16\u8bd1\u5230\u4e0a\u5c42\u5e94\u7528\u7684\u5168\u94fe\u8def\u5b9e\u8df5\uff0c\u662f\u975e\u5e38\u5b9d\u8d35\u7684\u7ecf\u9a8c\u3002\u60a8\u7684\u8bbe\u5907\u548c\u8fd9\u53f0\u672c\u5730 AI \u73af\u5883\uff0c\u5c06\u6210\u4e3a\u4f60\u5b66\u4e60\u3001\u79d1\u7814\u548c\u9879\u76ee\u5f00\u53d1\u7684\u5f97\u529b\u5de5\u5177\u3002<\/p>\n\n\n\n
      <\/p>\n","protected":false},"excerpt":{"rendered":"
      \u5f00\u7bc7\u5462\uff0c\u5148\u7ed9\u81ea\u5df1\u53e0\u4e2a\u7532\uff0c\u535a\u4e3b\u4e5f\u662f\u82b1\u4e86\u4e9b\u65f6\u95f4\u624d\u5b8c\u6210\uff0c\u6240\u4ee5\u522b\u5acc\u535a\u4e3b\u5570\u55e6\uff0c\u6211\u8ba4\u4e3a\u5570\u55e6\u624d\u662f\u4e00\u4e2a\u4eba\u5355\u72ec\u5b8c\u6210\u8be5\u90e8\u7f72\u7684\u72ec\u6709\u7559 […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45],"tags":[33,30,32,35,44],"class_list":["post-172","post","type-post","status-publish","format-standard","hentry","category-technical_sharing","tag-continue","tag-llama-cpp","tag-vscode","tag-35","tag-44"],"_links":{"self":[{"href":"https:\/\/www.chaohao.net.cn\/index.php?rest_route=\/wp\/v2\/posts\/172","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.chaohao.net.cn\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.chaohao.net.cn\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.chaohao.net.cn\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.chaohao.net.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=172"}],"version-history":[{"count":6,"href":"https:\/\/www.chaohao.net.cn\/index.php?rest_route=\/wp\/v2\/posts\/172\/revisions"}],"predecessor-version":[{"id":184,"href":"https:\/\/www.chaohao.net.cn\/index.php?rest_route=\/wp\/v2\/posts\/172\/revisions\/184"}],"wp:attachment":[{"href":"https:\/\/www.chaohao.net.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=172"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.chaohao.net.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=172"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.chaohao.net.cn\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=172"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}