Ollama支持OpenAI格式API的tool参数，在tool参数中，如果functions提供了参数，Qwen将会决定何时调用什么样的函数，不过Ollama目前还不支持强制使用特定函数的参数tool_choice。
注：本文测试用例参考OpenAI cookbook：https://cookbook.openai.com/examples/how_to_call_functions_with_chat_models
模型部署
GGUF 格式模型文件下载
使用ModelScope命令行工具下载单个模型，本文使用Qwen2-7B的GGUF格式：
modelscope download --model=qwen/Qwen2-7B-Instruct-GGUF --local_dir . qwen2-7b-instruct-q5_k_m.ggufLinux环境使用
Liunx用户可使用魔搭镜像环境安装【推荐】
modelscope download --model=modelscope/ollama-linux --local_dir ./ollama-linux
cd ollama-linux
sudo chmod 777 ./ollama-modelscope-install.sh
./ollama-modelscope-install.sh启动Ollama服务
ollama serve创建 ModelFile
复制模型路径，创建名为“ModelFile”的meta文件，其中设置template，使之支持function call，内容如下：
FROM /mnt/workspace/qwen2-7b-instruct-q5_k_m.gguf
# set the temperature to 0.7 [higher is more creative, lower is more coherent]
PARAMETER temperature 0.7
PARAMETER top_p 0.8
PARAMETER repeat_penalty 1.05
TEMPLATE """{{ if .Messages }}
{{- if or .System .Tools }}<|im_start|>system
{{ .System }}
{{- if .Tools }}
# Tools
You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. Here are the available tools:
<tools>{{- range .Tools }}{{ .Function }}{{- end }}</tools>
For each function call, return a JSON object with function name and arguments within <tool_call></tool_call> XML tags as follows:
<tool_call>
{"name": <function-name>, "arguments": <args-json-object>}
</tool_call>{{- end }}<|im_end|>{{- end }}
{{- range .Messages }}
{{- if eq .Role "user" }}
<|im_start|>{{ .Role }}
{{ .Content }}<|im_end|>
{{- else if eq .Role "assistant" }}
<|im_start|>{{ .Role }}
{{- if .Content }}
{{ .Content }}
{{- end }}
{{- if .ToolCalls }}
<tool_call>
{{ range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
{{ end }}</tool_call>
{{- end }}<|im_end|>
{{- else if eq .Role "tool" }}
<|im_start|>user
<tool_response>
{{ .Content }}
</tool_response><|im_end|>
{{- end }}
{{- end }}
<|im_start|>assistant
{{ else }}{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ end }}
"""创建自定义模型
使用ollama create命令创建自定义模型
ollama create myqwen2 --file ./ModelFile运行模型：
ollama run myqwen202
生成函数参数
安装依赖
!pip install scipy –quiet
!pip install tenacity –quiet
!pip install tiktoken –quiet
!pip install termcolor –quiet
!pip install openai –quiet
使用OpenAI的API格式调用本地部署的qwen2模型
import json
import openai
from tenacity import retry, wait_random_exponential, stop_after_attempt
from termcolor import colored
MODEL = “myqwen2”
client = openai.OpenAI(
base_url=”http://127.0.0.1:11434/v1",
api_key = “None”
)
实用工具
首先，让我们定义一些实用工具，用于调用聊天完成 API 以及维护和跟踪对话状态。
@retry(wait=wait_random_exponential(multiplier=1, max=40), stop=stop_after_attempt(3))
def chat_completion_request(messages, tools=None, tool_choice=None, model=MODEL):
try:
response = client.chat.completions.create(
model=model,
messages=messages,
tools=tools,
tool_choice=tool_choice,
)
return response
except Exception as e:
print("Unable to generate ChatCompletion response")
print(f"Exception: {e}")
return e
def pretty_print_conversation(messages):
role_to_color = {
"system": "red",
"user": "green",
"assistant": "blue",
"function": "magenta",
}
for message in messages:
if message["role"] == "system":
print(colored(f"system: {message['content']}\n", role_to_color[message["role"]]))
elif message["role"] == "user":
print(colored(f"user: {message['content']}\n", role_to_color[message["role"]]))
elif message["role"] == "assistant" and message.get("function_call"):
print(colored(f"assistant: {message['function_call']}\n", role_to_color[message["role"]]))
elif message["role"] == "assistant" and not message.get("function_call"):
print(colored(f"assistant: {message['content']}\n", role_to_color[message["role"]]))
elif message["role"] == "function":
print(colored(f"function ({message['name']}): {message['content']}\n", role_to_color[message["role"]]))基本概念（https://cookbook.openai.com/examples/how_to_call_functions_with_chat_models#basic-concepts）
这里假设了一个天气 API，并设置了一些函数规范和它进行交互。将这些函数规范传递给 Chat API，以便模型可以生成符合规范的函数参数。
tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"format": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The temperature unit to use. Infer this from the users location.",
},
},
"required": ["location", "format"],
},
}
},
{
"type": "function",
"function": {
"name": "get_n_day_weather_forecast",
"description": "Get an N-day weather forecast",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"format": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The temperature unit to use. Infer this from the users location.",
},
"num_days": {
"type": "integer",
"description": "The number of days to forecast",
}
},
"required": ["location", "format", "num_days"]
},
}
},
]如果我们向模型询问当前的天气情况，它将会反问，希望获取到进一步的更多的参数信息。
messages = []
messages.append({"role": "system", "content": "Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous."})
messages.append({"role": "user", "content": "hi ，can you tell me what's the weather like today"})
chat_response = chat_completion_request(
messages, tools=tools
)
assistant_message = chat_response.choices[0].message
messages.append(assistant_message)
assistant_message
ChatCompletionMessage(content='Of course, I can help with that. To provide accurate information, could you please specify the city and state you are interested in?', role='assistant', function_call=None, tool_calls=None)一旦我们通过对话提供缺失的参数信息，模型就会为我们生成适当的函数参数。
messages.append({"role": "user", "content": "I'm in Glasgow, Scotland."})
chat_response = chat_completion_request(
messages, tools=tools
)
assistant_message = chat_response.choices[0].message
messages.append(assistant_message)
assistant_message
ChatCompletionMessage(content='', role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_qq8e5z9w', function=Function(arguments='{"location":"Glasgow, Scotland"}', name='get_current_weather'), type='function')])通过不同的提示词，我们可以让它反问不同的问题以获取函数参数信息。
messages = []
messages.append({"role": "system", "content": "Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous."})
messages.append({"role": "user", "content": "can you tell me, what is the weather going to be like in Glasgow, Scotland in next x days"})
chat_response = chat_completion_request(
messages, tools=tools
)
assistant_message = chat_response.choices[0].message
messages.append(assistant_message)
assistant_message
ChatCompletionMessage(content='Sure, I can help with that. Could you please specify how many days ahead you want to know the weather forecast for Glasgow, Scotland?', role='assistant', function_call=None, tool_calls=None)
messages.append({"role": "user", "content": "5 days"})
chat_response = chat_completion_request(
messages, tools=tools
)
chat_response.choices[0]
Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='', role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_b7f3j7im', function=Function(arguments='{"location":"Glasgow, Scotland","num_days":5}', name='get_n_day_weather_forecast'), type='function')]))支持一次提问中，并行调用多次函数
messages = []
messages.append({"role": "system", "content": "Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous."})
messages.append({"role": "user", "content": "what is the weather going to be like in San Francisco and Glasgow over the next 4 days"})
chat_response = chat_completion_request(
messages, tools=tools, model=MODEL
)
assistant_message = chat_response.choices[0].message.tool_calls
assistant_message
[ChatCompletionMessageToolCall(id='call_vei89rz3', function=Function(arguments='{"location":"San Francisco, CA","num_days":4}', name='get_n_day_weather_forecast'), type='function'),
ChatCompletionMessageToolCall(id='call_4lgoubee', function=Function(arguments='{"location":"Glasgow, UK","num_days":4}', name='get_n_day_weather_forecast'), type='function')]
使用模型生成函数（https://cookbook.openai.com/examples/how_to_call_functions_with_chat_models#how-to-call-functions-with-model-generated-arguments）
在这个示例中，演示如何执行输入由模型生成的函数，并使用它来实现可以为我们解答有关数据库的问题的代理。
本文使用Chinook 示例数据库（https://www.sqlitetutorial.net/sqlite-sample-database/）。
指定执行 SQL 查询的函数（https://cookbook.openai.com/examples/how_to_call_functions_with_chat_models#specifying-a-function-to-execute-sql-queries）
首先，让我们定义一些有用的函数来从 SQLite 数据库中提取数据。
import sqlite3
conn = sqlite3.connect("data/Chinook.db")
print("Opened database successfully")
def get_table_names(conn):
"""Return a list of table names."""
table_names = []
tables = conn.execute("SELECT name FROM sqlite_master WHERE type='table';")
for table in tables.fetchall():
table_names.append(table[0])
return table_names
def get_column_names(conn, table_name):
"""Return a list of column names."""
column_names = []
columns = conn.execute(f"PRAGMA table_info('{table_name}');").fetchall()
for col in columns:
column_names.append(col[1])
return column_names
def get_database_info(conn):
"""Return a list of dicts containing the table name and columns for each table in the database."""
table_dicts = []
for table_name in get_table_names(conn):
columns_names = get_column_names(conn, table_name)
table_dicts.append({"table_name": table_name, "column_names": columns_names})
return table_dicts
现在可以使用这些实用函数来提取数据库模式的表示。
database_schema_dict = get_database_info(conn)
database_schema_string = "\n".join(
[
f"Table: {table['table_name']}\nColumns: {', '.join(table['column_names'])}"
for table in database_schema_dict
]
)
与之前一样，我们将为希望 API 为其生成参数的函数定义一个函数规范。请注意，我们正在将数据库模式插入到函数规范中。这对于模型了解这一点很重要。
tools = [
{
"type": "function",
"function": {
"name": "ask_database",
"description": "Use this function to answer user questions about music. Input should be a fully formed SQL query.",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": f"""
SQL query extracting info to answer the user's question.
SQL should be written using this database schema:
{database_schema_string}
The query should be returned in plain text, not in JSON.
""",
}
},
"required": ["query"],
},
}
}
]执行 SQL 查询（https://cookbook.openai.com/examples/how_to_call_functions_with_chat_models#executing-sql-queries）
现在让我们实现实际执行数据库查询的函数。
def ask_database(conn, query):
"""Function to query SQLite database with a provided SQL query."""
try:
results = str(conn.execute(query).fetchall())
except Exception as e:
results = f"query failed with error: {e}"
return results使用 Chat Completions API 调用函数的步骤：
（https://cookbook.openai.com/examples/how_to_call_functions_with_chat_models#steps-to-invoke-a-function-call-using-chat-completions-api）
步骤 1：向模型提示可能导致模型选择要使用的工具的内容。工具的描述（例如函数名称和签名）在“工具”列表中定义，并在 API 调用中传递给模型。如果选择，函数名称和参数将包含在响应中。
步骤 2：通过编程检查模型是否想要调用函数。如果是，则继续执行步骤 3。
步骤 3：从响应中提取函数名称和参数，使用参数调用该函数。将结果附加到消息中。
步骤 4：使用消息列表调用聊天完成 API 以获取响应。
messages = [{
"role":"user",
"content": "What is the name of the album with the most tracks?"
}]
response = client.chat.completions.create(
model='myqwen2',
messages=messages,
tools= tools,
tool_choice="auto"
)
# Append the message to messages list
response_message = response.choices[0].message
messages.append(response_message)
print(response_message)
ChatCompletionMessage(content='', role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_23nnhlv6', function=Function(arguments='{"query":"SELECT Album.Title FROM Album JOIN Track ON Album.AlbumId = Track.AlbumId GROUP BY Album.Title ORDER BY COUNT(*) DESC LIMIT 1"}', name='ask_database'), type='function')])
# Step 2: determine if the response from the model includes a tool call.
tool_calls = response_message.tool_calls
if tool_calls:
# If true the model will return the name of the tool / function to call and the argument(s)
tool_call_id = tool_calls[0].id
tool_function_name = tool_calls[0].function.name
tool_query_string = json.loads(tool_calls[0].function.arguments)['query']
# Step 3: Call the function and retrieve results. Append the results to the messages list.
if tool_function_name == 'ask_database':
results = ask_database(conn, tool_query_string)
messages.append({
"role":"tool",
"tool_call_id":tool_call_id,
"name": tool_function_name,
"content":results
})
# Step 4: Invoke the chat completions API with the function response appended to the messages list
# Note that messages with role 'tool' must be a response to a preceding message with 'tool_calls'
model_response_with_function_call = client.chat.completions.create(
model="myqwen2",
messages=messages,
) # get a new response from the model where it can see the function response
print(model_response_with_function_call.choices[0].message.content)
else:
print(f"Error: function {tool_function_name} does not exist")
else:
# Model did not identify a function to call, result can be returned to the user
print(response_message.content) The album “Greatest Hits” contains the most tracks
作者：Jeebiz 创建时间：2024-08-15 19:55
最后编辑：Jeebiz 更新时间：2024-08-29 20:41
最后编辑：Jeebiz 更新时间：2024-08-29 20:41
