Merge branch 'master' into improve_ui_master
This commit is contained in:
binary-husky
commit
73e835885b
2
.github/workflows/build-with-chatglm.yml
vendored
2
.github/workflows/build-with-chatglm.yml
vendored
@ -1,5 +1,5 @@
|
|||||||
# https://docs.github.com/en/actions/publishing-packages/publishing-docker-images#publishing-images-to-github-packages
|
# https://docs.github.com/en/actions/publishing-packages/publishing-docker-images#publishing-images-to-github-packages
|
||||||
name: Create and publish a Docker image for ChatGLM support
|
name: build-with-chatglm
|
||||||
|
|
||||||
on:
|
on:
|
||||||
push:
|
push:
|
||||||
|
|||||||
2
.github/workflows/build-with-latex.yml
vendored
2
.github/workflows/build-with-latex.yml
vendored
@ -1,5 +1,5 @@
|
|||||||
# https://docs.github.com/en/actions/publishing-packages/publishing-docker-images#publishing-images-to-github-packages
|
# https://docs.github.com/en/actions/publishing-packages/publishing-docker-images#publishing-images-to-github-packages
|
||||||
name: Create and publish a Docker image for Latex support
|
name: build-with-latex
|
||||||
|
|
||||||
on:
|
on:
|
||||||
push:
|
push:
|
||||||
|
|||||||
@ -1,5 +1,5 @@
|
|||||||
# https://docs.github.com/en/actions/publishing-packages/publishing-docker-images#publishing-images-to-github-packages
|
# https://docs.github.com/en/actions/publishing-packages/publishing-docker-images#publishing-images-to-github-packages
|
||||||
name: Create and publish a Docker image
|
name: build-without-local-llms
|
||||||
|
|
||||||
on:
|
on:
|
||||||
push:
|
push:
|
||||||
|
|||||||
31
README.md
31
README.md
@ -1,8 +1,6 @@
|
|||||||
> **Note**
|
> **Note**
|
||||||
>
|
>
|
||||||
> 2023.7.5: Gradio依赖调整。请及时**更新代码**
|
> 2023.7.8: Gradio, Pydantic依赖调整，已修改 `requirements.txt`。请及时**更新代码**，安装依赖时，请严格选择`requirements.txt`中**指定的版本**
|
||||||
>
|
|
||||||
> 2023.7.8: pydantic出现兼容问题，已修改 `requirements.txt`。安装依赖时，请严格选择`requirements.txt`中**指定的版本**
|
|
||||||
>
|
>
|
||||||
> `pip install -r requirements.txt`
|
> `pip install -r requirements.txt`
|
||||||
|
|
||||||
@ -43,15 +41,17 @@ Markdown[中英互译](https://www.bilibili.com/video/BV1yo4y157jV/) | [函数
|
|||||||
chat分析报告生成 | [函数插件] 运行后自动生成总结汇报
|
chat分析报告生成 | [函数插件] 运行后自动生成总结汇报
|
||||||
[PDF论文全文翻译功能](https://www.bilibili.com/video/BV1KT411x7Wn) | [函数插件] PDF论文提取题目&摘要+翻译全文（多线程）
|
[PDF论文全文翻译功能](https://www.bilibili.com/video/BV1KT411x7Wn) | [函数插件] PDF论文提取题目&摘要+翻译全文（多线程）
|
||||||
[Arxiv小助手](https://www.bilibili.com/video/BV1LM4y1279X) | [函数插件] 输入arxiv文章url即可一键翻译摘要+下载PDF
|
[Arxiv小助手](https://www.bilibili.com/video/BV1LM4y1279X) | [函数插件] 输入arxiv文章url即可一键翻译摘要+下载PDF
|
||||||
|
Latex论文一键校对 | [函数插件] 仿Grammarly对Latex文章进行语法、拼写纠错+输出对照PDF
|
||||||
[谷歌学术统合小助手](https://www.bilibili.com/video/BV19L411U7ia) | [函数插件] 给定任意谷歌学术搜索页面URL，让gpt帮你[写relatedworks](https://www.bilibili.com/video/BV1GP411U7Az/)
|
[谷歌学术统合小助手](https://www.bilibili.com/video/BV19L411U7ia) | [函数插件] 给定任意谷歌学术搜索页面URL，让gpt帮你[写relatedworks](https://www.bilibili.com/video/BV1GP411U7Az/)
|
||||||
互联网信息聚合+GPT | [函数插件] 一键[让GPT从互联网获取信息](https://www.bilibili.com/video/BV1om4y127ck)回答问题，让信息永不过时
|
互联网信息聚合+GPT | [函数插件] 一键[让GPT从互联网获取信息](https://www.bilibili.com/video/BV1om4y127ck)回答问题，让信息永不过时
|
||||||
⭐Arxiv论文精细翻译 | [函数插件] 一键[以超高质量翻译arxiv论文](https://www.bilibili.com/video/BV1dz4y1v77A/)，目前最好的论文翻译工具
|
⭐Arxiv论文精细翻译 | [函数插件] 一键[以超高质量翻译arxiv论文](https://www.bilibili.com/video/BV1dz4y1v77A/)，目前最好的论文翻译工具
|
||||||
公式/图片/表格显示 | 可以同时显示公式的[tex形式和渲染形式](https://user-images.githubusercontent.com/96192199/230598842-1d7fcddd-815d-40ee-af60-baf488a199df.png)，支持公式、代码高亮
|
公式/图片/表格显示 | 可以同时显示公式的[tex形式和渲染形式](https://user-images.githubusercontent.com/96192199/230598842-1d7fcddd-815d-40ee-af60-baf488a199df.png)，支持公式、代码高亮
|
||||||
多线程函数插件支持 | 支持多线调用chatgpt，一键处理[海量文本](https://www.bilibili.com/video/BV1FT411H7c5/)或程序
|
多线程函数插件支持 | 支持多线调用chatgpt，一键处理[海量文本](https://www.bilibili.com/video/BV1FT411H7c5/)或程序
|
||||||
启动暗色[主题](https://github.com/binary-husky/gpt_academic/issues/173) | 在浏览器url后面添加```/?__theme=dark```可以切换dark主题
|
启动暗色[主题](https://github.com/binary-husky/gpt_academic/issues/173) | 在浏览器url后面添加```/?__theme=dark```可以切换dark主题
|
||||||
[多LLM模型](https://www.bilibili.com/video/BV1wT411p7yf)支持 | 同时被GPT3.5、GPT4、[清华ChatGLM](https://github.com/THUDM/ChatGLM-6B)、[复旦MOSS](https://github.com/OpenLMLab/MOSS)同时伺候的感觉一定会很不错吧？
|
[多LLM模型](https://www.bilibili.com/video/BV1wT411p7yf)支持 | 同时被GPT3.5、GPT4、[清华ChatGLM2](https://github.com/THUDM/ChatGLM2-6B)、[复旦MOSS](https://github.com/OpenLMLab/MOSS)同时伺候的感觉一定会很不错吧？
|
||||||
|
ChatGLM2微调模型 | 支持加载ChatGLM2微调模型，提供ChatGLM2微调插件
|
||||||
更多LLM模型接入，支持[huggingface部署](https://huggingface.co/spaces/qingxu98/gpt-academic) | 加入Newbing接口(新必应)，引入清华[Jittorllms](https://github.com/Jittor/JittorLLMs)支持[LLaMA](https://github.com/facebookresearch/llama)和[盘古α](https://openi.org.cn/pangu/)
|
更多LLM模型接入，支持[huggingface部署](https://huggingface.co/spaces/qingxu98/gpt-academic) | 加入Newbing接口(新必应)，引入清华[Jittorllms](https://github.com/Jittor/JittorLLMs)支持[LLaMA](https://github.com/facebookresearch/llama)和[盘古α](https://openi.org.cn/pangu/)
|
||||||
更多新功能展示(图像生成等) …… | 见本文档结尾处 ……
|
更多新功能展示 (图像生成等) …… | 见本文档结尾处 ……
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
@ -115,12 +115,12 @@ python -m pip install -r requirements.txt # 这个步骤和pip安装一样的步
|
|||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
<details><summary>如果需要支持清华ChatGLM/复旦MOSS作为后端，请点击展开此处</summary>
|
<details><summary>如果需要支持清华ChatGLM2/复旦MOSS作为后端，请点击展开此处</summary>
|
||||||
<p>
|
<p>
|
||||||
|
|
||||||
【可选步骤】如果需要支持清华ChatGLM/复旦MOSS作为后端，需要额外安装更多依赖（前提条件：熟悉Python + 用过Pytorch + 电脑配置够强）：
|
【可选步骤】如果需要支持清华ChatGLM2/复旦MOSS作为后端，需要额外安装更多依赖（前提条件：熟悉Python + 用过Pytorch + 电脑配置够强）：
|
||||||
```sh
|
```sh
|
||||||
# 【可选步骤I】支持清华ChatGLM。清华ChatGLM备注：如果遇到"Call ChatGLM fail 不能正常加载ChatGLM的参数" 错误，参考如下： 1：以上默认安装的为torch+cpu版，使用cuda需要卸载torch重新安装torch+cuda； 2：如因本机配置不够无法加载模型，可以修改request_llm/bridge_chatglm.py中的模型精度, 将 AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True) 都修改为 AutoTokenizer.from_pretrained("THUDM/chatglm-6b-int4", trust_remote_code=True)
|
# 【可选步骤I】支持清华ChatGLM2。清华ChatGLM备注：如果遇到"Call ChatGLM fail 不能正常加载ChatGLM的参数" 错误，参考如下： 1：以上默认安装的为torch+cpu版，使用cuda需要卸载torch重新安装torch+cuda； 2：如因本机配置不够无法加载模型，可以修改request_llm/bridge_chatglm.py中的模型精度, 将 AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True) 都修改为 AutoTokenizer.from_pretrained("THUDM/chatglm-6b-int4", trust_remote_code=True)
|
||||||
python -m pip install -r request_llm/requirements_chatglm.txt
|
python -m pip install -r request_llm/requirements_chatglm.txt
|
||||||
|
|
||||||
# 【可选步骤II】支持复旦MOSS
|
# 【可选步骤II】支持复旦MOSS
|
||||||
@ -144,6 +144,8 @@ python main.py
|
|||||||
### 安装方法II：使用Docker
|
### 安装方法II：使用Docker
|
||||||
|
|
||||||
1. 仅ChatGPT（推荐大多数人选择，等价于docker-compose方案1）
|
1. 仅ChatGPT（推荐大多数人选择，等价于docker-compose方案1）
|
||||||
|
[](https://github.com/binary-husky/gpt_academic/actions/workflows/build-without-local-llms.yml)
|
||||||
|
[](https://github.com/binary-husky/gpt_academic/actions/workflows/build-with-latex.yml)
|
||||||
|
|
||||||
``` sh
|
``` sh
|
||||||
git clone https://github.com/binary-husky/gpt_academic.git # 下载项目
|
git clone https://github.com/binary-husky/gpt_academic.git # 下载项目
|
||||||
@ -151,14 +153,15 @@ cd gpt_academic # 进入路径
|
|||||||
nano config.py # 用任意文本编辑器编辑config.py, 配置 “Proxy”， “API_KEY” 以及 “WEB_PORT” (例如50923) 等
|
nano config.py # 用任意文本编辑器编辑config.py, 配置 “Proxy”， “API_KEY” 以及 “WEB_PORT” (例如50923) 等
|
||||||
docker build -t gpt-academic . # 安装
|
docker build -t gpt-academic . # 安装
|
||||||
|
|
||||||
#（最后一步-选择1）在Linux环境下，用`--net=host`更方便快捷
|
#（最后一步-Linux操作系统）用`--net=host`更方便快捷
|
||||||
docker run --rm -it --net=host gpt-academic
|
docker run --rm -it --net=host gpt-academic
|
||||||
#（最后一步-选择2）在macOS/windows环境下，只能用-p选项将容器上的端口(例如50923)暴露给主机上的端口
|
#（最后一步-MacOS/Windows操作系统）只能用-p选项将容器上的端口(例如50923)暴露给主机上的端口
|
||||||
docker run --rm -it -e WEB_PORT=50923 -p 50923:50923 gpt-academic
|
docker run --rm -it -e WEB_PORT=50923 -p 50923:50923 gpt-academic
|
||||||
```
|
```
|
||||||
P.S. 如果需要依赖Latex的插件功能，请见Wiki。另外，您也可以直接使用docker-compose获取Latex功能（修改docker-compose.yml，保留方案4并删除其他方案）。
|
P.S. 如果需要依赖Latex的插件功能，请见Wiki。另外，您也可以直接使用docker-compose获取Latex功能（修改docker-compose.yml，保留方案4并删除其他方案）。
|
||||||
|
|
||||||
2. ChatGPT + ChatGLM + MOSS（需要熟悉Docker）
|
2. ChatGPT + ChatGLM2 + MOSS（需要熟悉Docker）
|
||||||
|
[](https://github.com/binary-husky/gpt_academic/actions/workflows/build-with-chatglm.yml)
|
||||||
|
|
||||||
``` sh
|
``` sh
|
||||||
# 修改docker-compose.yml，保留方案2并删除其他方案。修改docker-compose.yml中方案2的配置，参考其中注释即可
|
# 修改docker-compose.yml，保留方案2并删除其他方案。修改docker-compose.yml中方案2的配置，参考其中注释即可
|
||||||
@ -284,6 +287,8 @@ Tip：不指定文件直接点击 `载入对话历史存档` 可以查看历史h
|
|||||||
|
|
||||||
### II：版本:
|
### II：版本:
|
||||||
- version 3.5(Todo): 使用自然语言调用本项目的所有函数插件（高优先级）
|
- version 3.5(Todo): 使用自然语言调用本项目的所有函数插件（高优先级）
|
||||||
|
- version 3.45: 支持自定义ChatGLM2微调模型
|
||||||
|
- version 3.44: 正式支持Azure，优化界面易用性
|
||||||
- version 3.4: +arxiv论文翻译、latex论文批改功能
|
- version 3.4: +arxiv论文翻译、latex论文批改功能
|
||||||
- version 3.3: +互联网信息综合功能
|
- version 3.3: +互联网信息综合功能
|
||||||
- version 3.2: 函数插件支持更多参数接口 (保存对话功能, 解读任意语言代码+同时询问任意的LLM组合)
|
- version 3.2: 函数插件支持更多参数接口 (保存对话功能, 解读任意语言代码+同时询问任意的LLM组合)
|
||||||
@ -314,8 +319,8 @@ gpt_academic开发者QQ群-2：610599535
|
|||||||
```
|
```
|
||||||
代码中参考了很多其他优秀项目中的设计，顺序不分先后：
|
代码中参考了很多其他优秀项目中的设计，顺序不分先后：
|
||||||
|
|
||||||
# 清华ChatGLM-6B:
|
# 清华ChatGLM2-6B:
|
||||||
https://github.com/THUDM/ChatGLM-6B
|
https://github.com/THUDM/ChatGLM2-6B
|
||||||
|
|
||||||
# 清华JittorLLMs:
|
# 清华JittorLLMs:
|
||||||
https://github.com/Jittor/JittorLLMs
|
https://github.com/Jittor/JittorLLMs
|
||||||
|
|||||||
@ -139,7 +139,7 @@ def auto_update(raise_error=False):
|
|||||||
else:
|
else:
|
||||||
return
|
return
|
||||||
except:
|
except:
|
||||||
msg = '自动更新程序：已禁用'
|
msg = '自动更新程序：已禁用。建议排查：代理网络配置。'
|
||||||
if raise_error:
|
if raise_error:
|
||||||
from toolbox import trimmed_format_exc
|
from toolbox import trimmed_format_exc
|
||||||
msg += trimmed_format_exc()
|
msg += trimmed_format_exc()
|
||||||
|
|||||||
@ -74,6 +74,10 @@ AVAIL_LLM_MODELS = ["gpt-3.5-turbo-16k", "gpt-3.5-turbo", "azure-gpt-3.5", "api2
|
|||||||
# P.S. 其他可用的模型还包括 ["gpt-3.5-turbo-0613", "gpt-3.5-turbo-16k-0613", "newbing-free", "jittorllms_rwkv", "jittorllms_pangualpha", "jittorllms_llama"]
|
# P.S. 其他可用的模型还包括 ["gpt-3.5-turbo-0613", "gpt-3.5-turbo-16k-0613", "newbing-free", "jittorllms_rwkv", "jittorllms_pangualpha", "jittorllms_llama"]
|
||||||
|
|
||||||
|
|
||||||
|
# ChatGLM(2) Finetune Model Path （如果使用ChatGLM2微调模型，需要把"chatglmft"加入AVAIL_LLM_MODELS中）
|
||||||
|
ChatGLM_PTUNING_CHECKPOINT = "" # 例如"/home/hmp/ChatGLM2-6B/ptuning/output/6b-pt-128-1e-2/checkpoint-100"
|
||||||
|
|
||||||
|
|
||||||
# 本地LLM模型如ChatGLM的执行方式 CPU/GPU
|
# 本地LLM模型如ChatGLM的执行方式 CPU/GPU
|
||||||
LOCAL_MODEL_DEVICE = "cpu" # 可选 "cuda"
|
LOCAL_MODEL_DEVICE = "cpu" # 可选 "cuda"
|
||||||
|
|
||||||
|
|||||||
@ -30,7 +30,7 @@ def 知识库问答(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_pro
|
|||||||
)
|
)
|
||||||
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
|
||||||
from .crazy_utils import try_install_deps
|
from .crazy_utils import try_install_deps
|
||||||
try_install_deps(['zh_langchain==0.2.1'])
|
try_install_deps(['zh_langchain==0.2.1', 'pypinyin'])
|
||||||
|
|
||||||
# < --------------------读取参数--------------- >
|
# < --------------------读取参数--------------- >
|
||||||
if ("advanced_arg" in plugin_kwargs) and (plugin_kwargs["advanced_arg"] == ""): plugin_kwargs.pop("advanced_arg")
|
if ("advanced_arg" in plugin_kwargs) and (plugin_kwargs["advanced_arg"] == ""): plugin_kwargs.pop("advanced_arg")
|
||||||
|
|||||||
@ -18,6 +18,13 @@ def string_to_options(arguments):
|
|||||||
parser.add_argument("--prompt_prefix", type=str, help="Prompt prefix", default='')
|
parser.add_argument("--prompt_prefix", type=str, help="Prompt prefix", default='')
|
||||||
parser.add_argument("--system_prompt", type=str, help="System prompt", default='')
|
parser.add_argument("--system_prompt", type=str, help="System prompt", default='')
|
||||||
parser.add_argument("--batch", type=int, help="System prompt", default=50)
|
parser.add_argument("--batch", type=int, help="System prompt", default=50)
|
||||||
|
parser.add_argument("--pre_seq_len", type=int, help="pre_seq_len", default=50)
|
||||||
|
parser.add_argument("--learning_rate", type=float, help="learning_rate", default=2e-2)
|
||||||
|
parser.add_argument("--num_gpus", type=int, help="num_gpus", default=1)
|
||||||
|
parser.add_argument("--json_dataset", type=str, help="json_dataset", default="")
|
||||||
|
parser.add_argument("--ptuning_directory", type=str, help="ptuning_directory", default="")
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
# Parse the arguments
|
# Parse the arguments
|
||||||
args = parser.parse_args(shlex.split(arguments))
|
args = parser.parse_args(shlex.split(arguments))
|
||||||
@ -69,3 +76,66 @@ def 微调数据集生成(txt, llm_kwargs, plugin_kwargs, chatbot, history, syst
|
|||||||
|
|
||||||
promote_file_to_downloadzone(txt+'.generated.json', rename_file='generated.json', chatbot=chatbot)
|
promote_file_to_downloadzone(txt+'.generated.json', rename_file='generated.json', chatbot=chatbot)
|
||||||
return
|
return
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
@CatchException
|
||||||
|
def 启动微调(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
|
||||||
|
"""
|
||||||
|
txt 输入栏用户输入的文本，例如需要翻译的一段话，再例如一个包含了待处理文件的路径
|
||||||
|
llm_kwargs gpt模型参数，如温度和top_p等，一般原样传递下去就行
|
||||||
|
plugin_kwargs 插件模型的参数
|
||||||
|
chatbot 聊天显示框的句柄，用于显示给用户
|
||||||
|
history 聊天历史，前情提要
|
||||||
|
system_prompt 给gpt的静默提醒
|
||||||
|
web_port 当前软件运行的端口号
|
||||||
|
"""
|
||||||
|
import subprocess
|
||||||
|
history = [] # 清空历史，以免输入溢出
|
||||||
|
chatbot.append(("这是什么功能？", "[Local Message] 微调数据集生成"))
|
||||||
|
if ("advanced_arg" in plugin_kwargs) and (plugin_kwargs["advanced_arg"] == ""): plugin_kwargs.pop("advanced_arg")
|
||||||
|
args = plugin_kwargs.get("advanced_arg", None)
|
||||||
|
if args is None:
|
||||||
|
chatbot.append(("没给定指令", "退出"))
|
||||||
|
yield from update_ui(chatbot=chatbot, history=history); return
|
||||||
|
else:
|
||||||
|
arguments = string_to_options(arguments=args)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
pre_seq_len = arguments.pre_seq_len # 128
|
||||||
|
learning_rate = arguments.learning_rate # 2e-2
|
||||||
|
num_gpus = arguments.num_gpus # 1
|
||||||
|
json_dataset = arguments.json_dataset # 't_code.json'
|
||||||
|
ptuning_directory = arguments.ptuning_directory # '/home/hmp/ChatGLM2-6B/ptuning'
|
||||||
|
|
||||||
|
command = f"torchrun --standalone --nnodes=1 --nproc-per-node={num_gpus} main.py \
|
||||||
|
--do_train \
|
||||||
|
--train_file AdvertiseGen/{json_dataset} \
|
||||||
|
--validation_file AdvertiseGen/{json_dataset} \
|
||||||
|
--preprocessing_num_workers 20 \
|
||||||
|
--prompt_column content \
|
||||||
|
--response_column summary \
|
||||||
|
--overwrite_cache \
|
||||||
|
--model_name_or_path THUDM/chatglm2-6b \
|
||||||
|
--output_dir output/clothgen-chatglm2-6b-pt-{pre_seq_len}-{learning_rate} \
|
||||||
|
--overwrite_output_dir \
|
||||||
|
--max_source_length 256 \
|
||||||
|
--max_target_length 256 \
|
||||||
|
--per_device_train_batch_size 1 \
|
||||||
|
--per_device_eval_batch_size 1 \
|
||||||
|
--gradient_accumulation_steps 16 \
|
||||||
|
--predict_with_generate \
|
||||||
|
--max_steps 100 \
|
||||||
|
--logging_steps 10 \
|
||||||
|
--save_steps 20 \
|
||||||
|
--learning_rate {learning_rate} \
|
||||||
|
--pre_seq_len {pre_seq_len} \
|
||||||
|
--quantization_bit 4"
|
||||||
|
|
||||||
|
process = subprocess.Popen(command, shell=True, cwd=ptuning_directory)
|
||||||
|
try:
|
||||||
|
process.communicate(timeout=3600*24)
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
process.kill()
|
||||||
|
return
|
||||||
|
|||||||
@ -212,11 +212,17 @@ def test_Latex():
|
|||||||
# cli_printer.print(cb) # print(cb)
|
# cli_printer.print(cb) # print(cb)
|
||||||
|
|
||||||
def test_chatglm_finetune():
|
def test_chatglm_finetune():
|
||||||
from crazy_functions.chatglm微调工具 import 微调数据集生成
|
from crazy_functions.chatglm微调工具 import 微调数据集生成, 启动微调
|
||||||
txt = 'build/dev.json'
|
txt = 'build/dev.json'
|
||||||
plugin_kwargs = {"advanced_arg":"--llm_to_learn=gpt-3.5-turbo --prompt_prefix='根据下面的服装类型提示，想象一个穿着者，对这个人外貌、身处的环境、内心世界、人设进行描写。要求：100字以内，用第二人称。' --system_prompt=''" }
|
plugin_kwargs = {"advanced_arg":"--llm_to_learn=gpt-3.5-turbo --prompt_prefix='根据下面的服装类型提示，想象一个穿着者，对这个人外貌、身处的环境、内心世界、人设进行描写。要求：100字以内，用第二人称。' --system_prompt=''" }
|
||||||
|
|
||||||
for cookies, cb, hist, msg in (微调数据集生成)(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
|
# for cookies, cb, hist, msg in (微调数据集生成)(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
|
||||||
|
# cli_printer.print(cb)
|
||||||
|
|
||||||
|
plugin_kwargs = {"advanced_arg":
|
||||||
|
" --pre_seq_len=128 --learning_rate=2e-2 --num_gpus=1 --json_dataset='t_code.json' --ptuning_directory='/home/hmp/ChatGLM2-6B/ptuning' " }
|
||||||
|
|
||||||
|
for cookies, cb, hist, msg in (启动微调)(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
|
||||||
cli_printer.print(cb)
|
cli_printer.print(cb)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@ -168,7 +168,8 @@ model_info = {
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
AVAIL_LLM_MODELS, = get_conf("AVAIL_LLM_MODELS")
|
AVAIL_LLM_MODELS, LLM_MODEL = get_conf("AVAIL_LLM_MODELS", "LLM_MODEL")
|
||||||
|
AVAIL_LLM_MODELS = AVAIL_LLM_MODELS + [LLM_MODEL]
|
||||||
if "jittorllms_rwkv" in AVAIL_LLM_MODELS:
|
if "jittorllms_rwkv" in AVAIL_LLM_MODELS:
|
||||||
from .bridge_jittorllms_rwkv import predict_no_ui_long_connection as rwkv_noui
|
from .bridge_jittorllms_rwkv import predict_no_ui_long_connection as rwkv_noui
|
||||||
from .bridge_jittorllms_rwkv import predict as rwkv_ui
|
from .bridge_jittorllms_rwkv import predict as rwkv_ui
|
||||||
@ -269,6 +270,24 @@ if "newbing" in AVAIL_LLM_MODELS: # same with newbing-free
|
|||||||
})
|
})
|
||||||
except:
|
except:
|
||||||
print(trimmed_format_exc())
|
print(trimmed_format_exc())
|
||||||
|
if "chatglmft" in AVAIL_LLM_MODELS: # same with newbing-free
|
||||||
|
try:
|
||||||
|
from .bridge_chatglmft import predict_no_ui_long_connection as chatglmft_noui
|
||||||
|
from .bridge_chatglmft import predict as chatglmft_ui
|
||||||
|
# claude
|
||||||
|
model_info.update({
|
||||||
|
"chatglmft": {
|
||||||
|
"fn_with_ui": chatglmft_ui,
|
||||||
|
"fn_without_ui": chatglmft_noui,
|
||||||
|
"endpoint": None,
|
||||||
|
"max_token": 4096,
|
||||||
|
"tokenizer": tokenizer_gpt35,
|
||||||
|
"token_cnt": get_token_num_gpt35,
|
||||||
|
}
|
||||||
|
})
|
||||||
|
except:
|
||||||
|
print(trimmed_format_exc())
|
||||||
|
|
||||||
|
|
||||||
def LLM_CATCH_EXCEPTION(f):
|
def LLM_CATCH_EXCEPTION(f):
|
||||||
"""
|
"""
|
||||||
@ -372,6 +391,6 @@ def predict(inputs, llm_kwargs, *args, **kwargs):
|
|||||||
additional_fn代表点击的哪个按钮，按钮见functional.py
|
additional_fn代表点击的哪个按钮，按钮见functional.py
|
||||||
"""
|
"""
|
||||||
|
|
||||||
method = model_info[llm_kwargs['llm_model']]["fn_with_ui"]
|
method = model_info[llm_kwargs['llm_model']]["fn_with_ui"] # 如果这里报错，检查config中的AVAIL_LLM_MODELS选项
|
||||||
yield from method(inputs, llm_kwargs, *args, **kwargs)
|
yield from method(inputs, llm_kwargs, *args, **kwargs)
|
||||||
|
|
||||||
|
|||||||
210
request_llm/bridge_chatglmft.py
Normal file
210
request_llm/bridge_chatglmft.py
Normal file
@ -0,0 +1,210 @@
|
|||||||
|
|
||||||
|
from transformers import AutoModel, AutoTokenizer
|
||||||
|
import time
|
||||||
|
import os
|
||||||
|
import json
|
||||||
|
import threading
|
||||||
|
import importlib
|
||||||
|
from toolbox import update_ui, get_conf
|
||||||
|
from multiprocessing import Process, Pipe
|
||||||
|
|
||||||
|
load_message = "ChatGLMFT尚未加载，加载需要一段时间。注意，取决于`config.py`的配置，ChatGLMFT消耗大量的内存（CPU）或显存（GPU），也许会导致低配计算机卡死 ……"
|
||||||
|
|
||||||
|
def string_to_options(arguments):
|
||||||
|
import argparse
|
||||||
|
import shlex
|
||||||
|
# Create an argparse.ArgumentParser instance
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
# Add command-line arguments
|
||||||
|
parser.add_argument("--llm_to_learn", type=str, help="LLM model to learn", default="gpt-3.5-turbo")
|
||||||
|
parser.add_argument("--prompt_prefix", type=str, help="Prompt prefix", default='')
|
||||||
|
parser.add_argument("--system_prompt", type=str, help="System prompt", default='')
|
||||||
|
parser.add_argument("--batch", type=int, help="System prompt", default=50)
|
||||||
|
# Parse the arguments
|
||||||
|
args = parser.parse_args(shlex.split(arguments))
|
||||||
|
return args
|
||||||
|
|
||||||
|
|
||||||
|
#################################################################################
|
||||||
|
class GetGLMFTHandle(Process):
|
||||||
|
def __init__(self):
|
||||||
|
super().__init__(daemon=True)
|
||||||
|
self.parent, self.child = Pipe()
|
||||||
|
self.chatglmft_model = None
|
||||||
|
self.chatglmft_tokenizer = None
|
||||||
|
self.info = ""
|
||||||
|
self.success = True
|
||||||
|
self.check_dependency()
|
||||||
|
self.start()
|
||||||
|
self.threadLock = threading.Lock()
|
||||||
|
|
||||||
|
def check_dependency(self):
|
||||||
|
try:
|
||||||
|
import sentencepiece
|
||||||
|
self.info = "依赖检测通过"
|
||||||
|
self.success = True
|
||||||
|
except:
|
||||||
|
self.info = "缺少ChatGLMFT的依赖，如果要使用ChatGLMFT，除了基础的pip依赖以外，您还需要运行`pip install -r request_llm/requirements_chatglm.txt`安装ChatGLM的依赖。"
|
||||||
|
self.success = False
|
||||||
|
|
||||||
|
def ready(self):
|
||||||
|
return self.chatglmft_model is not None
|
||||||
|
|
||||||
|
def run(self):
|
||||||
|
# 子进程执行
|
||||||
|
# 第一次运行，加载参数
|
||||||
|
retry = 0
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
if self.chatglmft_model is None:
|
||||||
|
from transformers import AutoConfig
|
||||||
|
import torch
|
||||||
|
# conf = 'request_llm/current_ptune_model.json'
|
||||||
|
# if not os.path.exists(conf): raise RuntimeError('找不到微调模型信息')
|
||||||
|
# with open(conf, 'r', encoding='utf8') as f:
|
||||||
|
# model_args = json.loads(f.read())
|
||||||
|
ChatGLM_PTUNING_CHECKPOINT, = get_conf('ChatGLM_PTUNING_CHECKPOINT')
|
||||||
|
assert os.path.exists(ChatGLM_PTUNING_CHECKPOINT), "找不到微调模型检查点"
|
||||||
|
conf = os.path.join(ChatGLM_PTUNING_CHECKPOINT, "config.json")
|
||||||
|
with open(conf, 'r', encoding='utf8') as f:
|
||||||
|
model_args = json.loads(f.read())
|
||||||
|
if 'model_name_or_path' not in model_args:
|
||||||
|
model_args['model_name_or_path'] = model_args['_name_or_path']
|
||||||
|
self.chatglmft_tokenizer = AutoTokenizer.from_pretrained(
|
||||||
|
model_args['model_name_or_path'], trust_remote_code=True)
|
||||||
|
config = AutoConfig.from_pretrained(
|
||||||
|
model_args['model_name_or_path'], trust_remote_code=True)
|
||||||
|
|
||||||
|
config.pre_seq_len = model_args['pre_seq_len']
|
||||||
|
config.prefix_projection = model_args['prefix_projection']
|
||||||
|
|
||||||
|
print(f"Loading prefix_encoder weight from {ChatGLM_PTUNING_CHECKPOINT}")
|
||||||
|
model = AutoModel.from_pretrained(model_args['model_name_or_path'], config=config, trust_remote_code=True)
|
||||||
|
prefix_state_dict = torch.load(os.path.join(ChatGLM_PTUNING_CHECKPOINT, "pytorch_model.bin"))
|
||||||
|
new_prefix_state_dict = {}
|
||||||
|
for k, v in prefix_state_dict.items():
|
||||||
|
if k.startswith("transformer.prefix_encoder."):
|
||||||
|
new_prefix_state_dict[k[len("transformer.prefix_encoder."):]] = v
|
||||||
|
model.transformer.prefix_encoder.load_state_dict(new_prefix_state_dict)
|
||||||
|
|
||||||
|
if model_args['quantization_bit'] is not None:
|
||||||
|
print(f"Quantized to {model_args['quantization_bit']} bit")
|
||||||
|
model = model.quantize(model_args['quantization_bit'])
|
||||||
|
model = model.cuda()
|
||||||
|
if model_args['pre_seq_len'] is not None:
|
||||||
|
# P-tuning v2
|
||||||
|
model.transformer.prefix_encoder.float()
|
||||||
|
self.chatglmft_model = model.eval()
|
||||||
|
|
||||||
|
break
|
||||||
|
else:
|
||||||
|
break
|
||||||
|
except Exception as e:
|
||||||
|
retry += 1
|
||||||
|
if retry > 3:
|
||||||
|
self.child.send('[Local Message] Call ChatGLMFT fail 不能正常加载ChatGLMFT的参数。')
|
||||||
|
raise RuntimeError("不能正常加载ChatGLMFT的参数！")
|
||||||
|
|
||||||
|
while True:
|
||||||
|
# 进入任务等待状态
|
||||||
|
kwargs = self.child.recv()
|
||||||
|
# 收到消息，开始请求
|
||||||
|
try:
|
||||||
|
for response, history in self.chatglmft_model.stream_chat(self.chatglmft_tokenizer, **kwargs):
|
||||||
|
self.child.send(response)
|
||||||
|
# # 中途接收可能的终止指令（如果有的话）
|
||||||
|
# if self.child.poll():
|
||||||
|
# command = self.child.recv()
|
||||||
|
# if command == '[Terminate]': break
|
||||||
|
except:
|
||||||
|
from toolbox import trimmed_format_exc
|
||||||
|
self.child.send('[Local Message] Call ChatGLMFT fail.' + '\n```\n' + trimmed_format_exc() + '\n```\n')
|
||||||
|
# 请求处理结束，开始下一个循环
|
||||||
|
self.child.send('[Finish]')
|
||||||
|
|
||||||
|
def stream_chat(self, **kwargs):
|
||||||
|
# 主进程执行
|
||||||
|
self.threadLock.acquire()
|
||||||
|
self.parent.send(kwargs)
|
||||||
|
while True:
|
||||||
|
res = self.parent.recv()
|
||||||
|
if res != '[Finish]':
|
||||||
|
yield res
|
||||||
|
else:
|
||||||
|
break
|
||||||
|
self.threadLock.release()
|
||||||
|
|
||||||
|
global glmft_handle
|
||||||
|
glmft_handle = None
|
||||||
|
#################################################################################
|
||||||
|
def predict_no_ui_long_connection(inputs, llm_kwargs, history=[], sys_prompt="", observe_window=[], console_slience=False):
|
||||||
|
"""
|
||||||
|
多线程方法
|
||||||
|
函数的说明请见 request_llm/bridge_all.py
|
||||||
|
"""
|
||||||
|
global glmft_handle
|
||||||
|
if glmft_handle is None:
|
||||||
|
glmft_handle = GetGLMFTHandle()
|
||||||
|
if len(observe_window) >= 1: observe_window[0] = load_message + "\n\n" + glmft_handle.info
|
||||||
|
if not glmft_handle.success:
|
||||||
|
error = glmft_handle.info
|
||||||
|
glmft_handle = None
|
||||||
|
raise RuntimeError(error)
|
||||||
|
|
||||||
|
# chatglmft 没有 sys_prompt 接口，因此把prompt加入 history
|
||||||
|
history_feedin = []
|
||||||
|
history_feedin.append(["What can I do?", sys_prompt])
|
||||||
|
for i in range(len(history)//2):
|
||||||
|
history_feedin.append([history[2*i], history[2*i+1]] )
|
||||||
|
|
||||||
|
watch_dog_patience = 5 # 看门狗 (watchdog) 的耐心, 设置5秒即可
|
||||||
|
response = ""
|
||||||
|
for response in glmft_handle.stream_chat(query=inputs, history=history_feedin, max_length=llm_kwargs['max_length'], top_p=llm_kwargs['top_p'], temperature=llm_kwargs['temperature']):
|
||||||
|
if len(observe_window) >= 1: observe_window[0] = response
|
||||||
|
if len(observe_window) >= 2:
|
||||||
|
if (time.time()-observe_window[1]) > watch_dog_patience:
|
||||||
|
raise RuntimeError("程序终止。")
|
||||||
|
return response
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
def predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_prompt='', stream = True, additional_fn=None):
|
||||||
|
"""
|
||||||
|
单线程方法
|
||||||
|
函数的说明请见 request_llm/bridge_all.py
|
||||||
|
"""
|
||||||
|
chatbot.append((inputs, ""))
|
||||||
|
|
||||||
|
global glmft_handle
|
||||||
|
if glmft_handle is None:
|
||||||
|
glmft_handle = GetGLMFTHandle()
|
||||||
|
chatbot[-1] = (inputs, load_message + "\n\n" + glmft_handle.info)
|
||||||
|
yield from update_ui(chatbot=chatbot, history=[])
|
||||||
|
if not glmft_handle.success:
|
||||||
|
glmft_handle = None
|
||||||
|
return
|
||||||
|
|
||||||
|
if additional_fn is not None:
|
||||||
|
import core_functional
|
||||||
|
importlib.reload(core_functional) # 热更新prompt
|
||||||
|
core_functional = core_functional.get_core_functions()
|
||||||
|
if "PreProcess" in core_functional[additional_fn]: inputs = core_functional[additional_fn]["PreProcess"](inputs) # 获取预处理函数（如果有的话）
|
||||||
|
inputs = core_functional[additional_fn]["Prefix"] + inputs + core_functional[additional_fn]["Suffix"]
|
||||||
|
|
||||||
|
# 处理历史信息
|
||||||
|
history_feedin = []
|
||||||
|
history_feedin.append(["What can I do?", system_prompt] )
|
||||||
|
for i in range(len(history)//2):
|
||||||
|
history_feedin.append([history[2*i], history[2*i+1]] )
|
||||||
|
|
||||||
|
# 开始接收chatglmft的回复
|
||||||
|
response = "[Local Message]: 等待ChatGLMFT响应中 ..."
|
||||||
|
for response in glmft_handle.stream_chat(query=inputs, history=history_feedin, max_length=llm_kwargs['max_length'], top_p=llm_kwargs['top_p'], temperature=llm_kwargs['temperature']):
|
||||||
|
chatbot[-1] = (inputs, response)
|
||||||
|
yield from update_ui(chatbot=chatbot, history=history)
|
||||||
|
|
||||||
|
# 总结输出
|
||||||
|
if response == "[Local Message]: 等待ChatGLMFT响应中 ...":
|
||||||
|
response = "[Local Message]: ChatGLMFT响应异常 ..."
|
||||||
|
history.extend([inputs, response])
|
||||||
|
yield from update_ui(chatbot=chatbot, history=history)
|
||||||
4
version
4
version
@ -1,5 +1,5 @@
|
|||||||
{
|
{
|
||||||
"version": 3.44,
|
"version": 3.45,
|
||||||
"show_feature": true,
|
"show_feature": true,
|
||||||
"new_feature": "[改善UI] 动态ChatBot窗口高度 <-> 修复Azure接口的BUG <-> 完善多语言模块 <-> 完善本地Latex矫错和翻译功能 <-> 增加gpt-3.5-16k的支持 <-> 新增最强Arxiv论文翻译插件 <-> 修复gradio复制按钮BUG <-> 修复PDF翻译的BUG, 新增HTML中英双栏对照 <-> 添加了OpenAI图片生成插件"
|
"new_feature": "支持加载自定义的ChatGLM2微调模型 <-> [改善UI] 动态ChatBot窗口高度 <-> 修复Azure接口的BUG <-> 完善多语言模块 <-> 完善本地Latex矫错和翻译功能 <-> 增加gpt-3.5-16k的支持 <-> 新增最强Arxiv论文翻译插件 <-> 修复gradio复制按钮BUG <-> 修复PDF翻译的BUG, 新增HTML中英双栏对照 <-> 添加了OpenAI图片生成插件"
|
||||||
}
|
}
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user