下载模型

https://ollama.com/library

ollama run llama2

带指标的运行

ollama run llama3:instruct --verbose

>>> 你好
😊 你好！我也很高兴地见到你！有什么问题或想聊天吗？ 🤗

total duration:       3.7415187s
load duration:        1.0918ms
prompt eval count:    12 token(s)
prompt eval duration: 801.813ms
prompt eval rate:     14.97 tokens/s
eval count:           27 token(s)
eval duration:        2.936699s
eval rate:            9.19 tokens/s

修改默认模型

Ollama 默认提供 OpenAI 的兼容 API，默认端口是 11434，默认模型名是 run 后面使用的模型名，如果想使用 OpenAI 的模型名，可以通过ollama cp的方式，为模型起一个临时的名称。

例如，

ollama cp mixtral:8x7b-instruct-v0.1-fp16 gpt-4
ollama run gpt-4

查看模型

ollama ls

Python接口对话

import ollama
 
response = ollama.chat(model='llama2', messages=[
  {
    'role': 'user',
    'content': 'Why is the sky blue?',
  },
])
print(response['message']['content'])

OpenAI接口对话

from openai import OpenAI
 
client = OpenAI(
    base_url = 'http://localhost:11434/v1',
    api_key='ollama', # required, but unused
)
 
response = client.chat.completions.create(
  model="llama2",
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Who won the world series in 2020?"},
    {"role": "assistant", "content": "The LA Dodgers won in 2020."},
    {"role": "user", "content": "Where was it played?"}
  ]
)
print(response.choices[0].message.content)

流式接口对话

curl -X POST http://localhost:11434/api/generate -d '{
  "model": "llama2",
  "prompt":"Why is the sky blue?"
 }'