diff --git a/.github/workflows/build-with-chatglm.yml b/.github/workflows/build-with-chatglm.yml index f968bb9..f729abb 100644 --- a/.github/workflows/build-with-chatglm.yml +++ b/.github/workflows/build-with-chatglm.yml @@ -1,5 +1,5 @@ # https://docs.github.com/en/actions/publishing-packages/publishing-docker-images#publishing-images-to-github-packages -name: Create and publish a Docker image for ChatGLM support +name: build-with-chatglm on: push: diff --git a/.github/workflows/build-with-latex.yml b/.github/workflows/build-with-latex.yml index fb16d2c..173d482 100644 --- a/.github/workflows/build-with-latex.yml +++ b/.github/workflows/build-with-latex.yml @@ -1,5 +1,5 @@ # https://docs.github.com/en/actions/publishing-packages/publishing-docker-images#publishing-images-to-github-packages -name: Create and publish a Docker image for Latex support +name: build-with-latex on: push: diff --git a/.github/workflows/build-without-local-llms.yml b/.github/workflows/build-without-local-llms.yml index b0aed7f..7bebd06 100644 --- a/.github/workflows/build-without-local-llms.yml +++ b/.github/workflows/build-without-local-llms.yml @@ -1,5 +1,5 @@ # https://docs.github.com/en/actions/publishing-packages/publishing-docker-images#publishing-images-to-github-packages -name: Create and publish a Docker image +name: build-without-local-llms on: push: diff --git a/README.md b/README.md index 6dd2cfa..b6a31bf 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,6 @@ > **Note** > -> 2023.7.5: Gradio依赖调整。请及时**更新代码** -> -> 2023.7.8: pydantic出现兼容问题,已修改 `requirements.txt`。安装依赖时,请严格选择`requirements.txt`中**指定的版本** +> 2023.7.8: Gradio, Pydantic依赖调整,已修改 `requirements.txt`。请及时**更新代码**,安装依赖时,请严格选择`requirements.txt`中**指定的版本** > > `pip install -r requirements.txt` @@ -43,15 +41,17 @@ Markdown[中英互译](https://www.bilibili.com/video/BV1yo4y157jV/) | [函数 chat分析报告生成 | [函数插件] 运行后自动生成总结汇报 [PDF论文全文翻译功能](https://www.bilibili.com/video/BV1KT411x7Wn) | [函数插件] PDF论文提取题目&摘要+翻译全文(多线程) [Arxiv小助手](https://www.bilibili.com/video/BV1LM4y1279X) | [函数插件] 输入arxiv文章url即可一键翻译摘要+下载PDF +Latex论文一键校对 | [函数插件] 仿Grammarly对Latex文章进行语法、拼写纠错+输出对照PDF [谷歌学术统合小助手](https://www.bilibili.com/video/BV19L411U7ia) | [函数插件] 给定任意谷歌学术搜索页面URL,让gpt帮你[写relatedworks](https://www.bilibili.com/video/BV1GP411U7Az/) 互联网信息聚合+GPT | [函数插件] 一键[让GPT从互联网获取信息](https://www.bilibili.com/video/BV1om4y127ck)回答问题,让信息永不过时 ⭐Arxiv论文精细翻译 | [函数插件] 一键[以超高质量翻译arxiv论文](https://www.bilibili.com/video/BV1dz4y1v77A/),目前最好的论文翻译工具 公式/图片/表格显示 | 可以同时显示公式的[tex形式和渲染形式](https://user-images.githubusercontent.com/96192199/230598842-1d7fcddd-815d-40ee-af60-baf488a199df.png),支持公式、代码高亮 多线程函数插件支持 | 支持多线调用chatgpt,一键处理[海量文本](https://www.bilibili.com/video/BV1FT411H7c5/)或程序 启动暗色[主题](https://github.com/binary-husky/gpt_academic/issues/173) | 在浏览器url后面添加```/?__theme=dark```可以切换dark主题 -[多LLM模型](https://www.bilibili.com/video/BV1wT411p7yf)支持 | 同时被GPT3.5、GPT4、[清华ChatGLM](https://github.com/THUDM/ChatGLM-6B)、[复旦MOSS](https://github.com/OpenLMLab/MOSS)同时伺候的感觉一定会很不错吧? +[多LLM模型](https://www.bilibili.com/video/BV1wT411p7yf)支持 | 同时被GPT3.5、GPT4、[清华ChatGLM2](https://github.com/THUDM/ChatGLM2-6B)、[复旦MOSS](https://github.com/OpenLMLab/MOSS)同时伺候的感觉一定会很不错吧? +ChatGLM2微调模型 | 支持加载ChatGLM2微调模型,提供ChatGLM2微调插件 更多LLM模型接入,支持[huggingface部署](https://huggingface.co/spaces/qingxu98/gpt-academic) | 加入Newbing接口(新必应),引入清华[Jittorllms](https://github.com/Jittor/JittorLLMs)支持[LLaMA](https://github.com/facebookresearch/llama)和[盘古α](https://openi.org.cn/pangu/) -更多新功能展示(图像生成等) …… | 见本文档结尾处 …… +更多新功能展示 (图像生成等) …… | 见本文档结尾处 …… @@ -115,12 +115,12 @@ python -m pip install -r requirements.txt # 这个步骤和pip安装一样的步 ``` -
如果需要支持清华ChatGLM/复旦MOSS作为后端,请点击展开此处 +
如果需要支持清华ChatGLM2/复旦MOSS作为后端,请点击展开此处

-【可选步骤】如果需要支持清华ChatGLM/复旦MOSS作为后端,需要额外安装更多依赖(前提条件:熟悉Python + 用过Pytorch + 电脑配置够强): +【可选步骤】如果需要支持清华ChatGLM2/复旦MOSS作为后端,需要额外安装更多依赖(前提条件:熟悉Python + 用过Pytorch + 电脑配置够强): ```sh -# 【可选步骤I】支持清华ChatGLM。清华ChatGLM备注:如果遇到"Call ChatGLM fail 不能正常加载ChatGLM的参数" 错误,参考如下: 1:以上默认安装的为torch+cpu版,使用cuda需要卸载torch重新安装torch+cuda; 2:如因本机配置不够无法加载模型,可以修改request_llm/bridge_chatglm.py中的模型精度, 将 AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True) 都修改为 AutoTokenizer.from_pretrained("THUDM/chatglm-6b-int4", trust_remote_code=True) +# 【可选步骤I】支持清华ChatGLM2。清华ChatGLM备注:如果遇到"Call ChatGLM fail 不能正常加载ChatGLM的参数" 错误,参考如下: 1:以上默认安装的为torch+cpu版,使用cuda需要卸载torch重新安装torch+cuda; 2:如因本机配置不够无法加载模型,可以修改request_llm/bridge_chatglm.py中的模型精度, 将 AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True) 都修改为 AutoTokenizer.from_pretrained("THUDM/chatglm-6b-int4", trust_remote_code=True) python -m pip install -r request_llm/requirements_chatglm.txt # 【可选步骤II】支持复旦MOSS @@ -144,6 +144,8 @@ python main.py ### 安装方法II:使用Docker 1. 仅ChatGPT(推荐大多数人选择,等价于docker-compose方案1) +[![basic](https://github.com/binary-husky/gpt_academic/actions/workflows/build-without-local-llms.yml/badge.svg?branch=master)](https://github.com/binary-husky/gpt_academic/actions/workflows/build-without-local-llms.yml) +[![basic](https://github.com/binary-husky/gpt_academic/actions/workflows/build-with-latex.yml/badge.svg?branch=master)](https://github.com/binary-husky/gpt_academic/actions/workflows/build-with-latex.yml) ``` sh git clone https://github.com/binary-husky/gpt_academic.git # 下载项目 @@ -151,14 +153,15 @@ cd gpt_academic # 进入路径 nano config.py # 用任意文本编辑器编辑config.py, 配置 “Proxy”, “API_KEY” 以及 “WEB_PORT” (例如50923) 等 docker build -t gpt-academic . # 安装 -#(最后一步-选择1)在Linux环境下,用`--net=host`更方便快捷 +#(最后一步-Linux操作系统)用`--net=host`更方便快捷 docker run --rm -it --net=host gpt-academic -#(最后一步-选择2)在macOS/windows环境下,只能用-p选项将容器上的端口(例如50923)暴露给主机上的端口 +#(最后一步-MacOS/Windows操作系统)只能用-p选项将容器上的端口(例如50923)暴露给主机上的端口 docker run --rm -it -e WEB_PORT=50923 -p 50923:50923 gpt-academic ``` P.S. 如果需要依赖Latex的插件功能,请见Wiki。另外,您也可以直接使用docker-compose获取Latex功能(修改docker-compose.yml,保留方案4并删除其他方案)。 -2. ChatGPT + ChatGLM + MOSS(需要熟悉Docker) +2. ChatGPT + ChatGLM2 + MOSS(需要熟悉Docker) +[![chatglm](https://github.com/binary-husky/gpt_academic/actions/workflows/build-with-chatglm.yml/badge.svg?branch=master)](https://github.com/binary-husky/gpt_academic/actions/workflows/build-with-chatglm.yml) ``` sh # 修改docker-compose.yml,保留方案2并删除其他方案。修改docker-compose.yml中方案2的配置,参考其中注释即可 @@ -284,6 +287,8 @@ Tip:不指定文件直接点击 `载入对话历史存档` 可以查看历史h ### II:版本: - version 3.5(Todo): 使用自然语言调用本项目的所有函数插件(高优先级) +- version 3.45: 支持自定义ChatGLM2微调模型 +- version 3.44: 正式支持Azure,优化界面易用性 - version 3.4: +arxiv论文翻译、latex论文批改功能 - version 3.3: +互联网信息综合功能 - version 3.2: 函数插件支持更多参数接口 (保存对话功能, 解读任意语言代码+同时询问任意的LLM组合) @@ -314,8 +319,8 @@ gpt_academic开发者QQ群-2:610599535 ``` 代码中参考了很多其他优秀项目中的设计,顺序不分先后: -# 清华ChatGLM-6B: -https://github.com/THUDM/ChatGLM-6B +# 清华ChatGLM2-6B: +https://github.com/THUDM/ChatGLM2-6B # 清华JittorLLMs: https://github.com/Jittor/JittorLLMs diff --git a/check_proxy.py b/check_proxy.py index eacf2c6..964ae47 100644 --- a/check_proxy.py +++ b/check_proxy.py @@ -139,7 +139,7 @@ def auto_update(raise_error=False): else: return except: - msg = '自动更新程序:已禁用' + msg = '自动更新程序:已禁用。建议排查:代理网络配置。' if raise_error: from toolbox import trimmed_format_exc msg += trimmed_format_exc() diff --git a/config.py b/config.py index 4fbb331..150fffd 100644 --- a/config.py +++ b/config.py @@ -74,6 +74,10 @@ AVAIL_LLM_MODELS = ["gpt-3.5-turbo-16k", "gpt-3.5-turbo", "azure-gpt-3.5", "api2 # P.S. 其他可用的模型还包括 ["gpt-3.5-turbo-0613", "gpt-3.5-turbo-16k-0613", "newbing-free", "jittorllms_rwkv", "jittorllms_pangualpha", "jittorllms_llama"] +# ChatGLM(2) Finetune Model Path (如果使用ChatGLM2微调模型,需要把"chatglmft"加入AVAIL_LLM_MODELS中) +ChatGLM_PTUNING_CHECKPOINT = "" # 例如"/home/hmp/ChatGLM2-6B/ptuning/output/6b-pt-128-1e-2/checkpoint-100" + + # 本地LLM模型如ChatGLM的执行方式 CPU/GPU LOCAL_MODEL_DEVICE = "cpu" # 可选 "cuda" diff --git a/crazy_functions/Langchain知识库.py b/crazy_functions/Langchain知识库.py index 31c459a..12735df 100644 --- a/crazy_functions/Langchain知识库.py +++ b/crazy_functions/Langchain知识库.py @@ -30,7 +30,7 @@ def 知识库问答(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_pro ) yield from update_ui(chatbot=chatbot, history=history) # 刷新界面 from .crazy_utils import try_install_deps - try_install_deps(['zh_langchain==0.2.1']) + try_install_deps(['zh_langchain==0.2.1', 'pypinyin']) # < --------------------读取参数--------------- > if ("advanced_arg" in plugin_kwargs) and (plugin_kwargs["advanced_arg"] == ""): plugin_kwargs.pop("advanced_arg") diff --git a/crazy_functions/chatglm微调工具.py b/crazy_functions/chatglm微调工具.py index 58a9208..336d7cf 100644 --- a/crazy_functions/chatglm微调工具.py +++ b/crazy_functions/chatglm微调工具.py @@ -18,6 +18,13 @@ def string_to_options(arguments): parser.add_argument("--prompt_prefix", type=str, help="Prompt prefix", default='') parser.add_argument("--system_prompt", type=str, help="System prompt", default='') parser.add_argument("--batch", type=int, help="System prompt", default=50) + parser.add_argument("--pre_seq_len", type=int, help="pre_seq_len", default=50) + parser.add_argument("--learning_rate", type=float, help="learning_rate", default=2e-2) + parser.add_argument("--num_gpus", type=int, help="num_gpus", default=1) + parser.add_argument("--json_dataset", type=str, help="json_dataset", default="") + parser.add_argument("--ptuning_directory", type=str, help="ptuning_directory", default="") + + # Parse the arguments args = parser.parse_args(shlex.split(arguments)) @@ -69,3 +76,66 @@ def 微调数据集生成(txt, llm_kwargs, plugin_kwargs, chatbot, history, syst promote_file_to_downloadzone(txt+'.generated.json', rename_file='generated.json', chatbot=chatbot) return + + + +@CatchException +def 启动微调(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port): + """ + txt 输入栏用户输入的文本,例如需要翻译的一段话,再例如一个包含了待处理文件的路径 + llm_kwargs gpt模型参数,如温度和top_p等,一般原样传递下去就行 + plugin_kwargs 插件模型的参数 + chatbot 聊天显示框的句柄,用于显示给用户 + history 聊天历史,前情提要 + system_prompt 给gpt的静默提醒 + web_port 当前软件运行的端口号 + """ + import subprocess + history = [] # 清空历史,以免输入溢出 + chatbot.append(("这是什么功能?", "[Local Message] 微调数据集生成")) + if ("advanced_arg" in plugin_kwargs) and (plugin_kwargs["advanced_arg"] == ""): plugin_kwargs.pop("advanced_arg") + args = plugin_kwargs.get("advanced_arg", None) + if args is None: + chatbot.append(("没给定指令", "退出")) + yield from update_ui(chatbot=chatbot, history=history); return + else: + arguments = string_to_options(arguments=args) + + + + pre_seq_len = arguments.pre_seq_len # 128 + learning_rate = arguments.learning_rate # 2e-2 + num_gpus = arguments.num_gpus # 1 + json_dataset = arguments.json_dataset # 't_code.json' + ptuning_directory = arguments.ptuning_directory # '/home/hmp/ChatGLM2-6B/ptuning' + + command = f"torchrun --standalone --nnodes=1 --nproc-per-node={num_gpus} main.py \ + --do_train \ + --train_file AdvertiseGen/{json_dataset} \ + --validation_file AdvertiseGen/{json_dataset} \ + --preprocessing_num_workers 20 \ + --prompt_column content \ + --response_column summary \ + --overwrite_cache \ + --model_name_or_path THUDM/chatglm2-6b \ + --output_dir output/clothgen-chatglm2-6b-pt-{pre_seq_len}-{learning_rate} \ + --overwrite_output_dir \ + --max_source_length 256 \ + --max_target_length 256 \ + --per_device_train_batch_size 1 \ + --per_device_eval_batch_size 1 \ + --gradient_accumulation_steps 16 \ + --predict_with_generate \ + --max_steps 100 \ + --logging_steps 10 \ + --save_steps 20 \ + --learning_rate {learning_rate} \ + --pre_seq_len {pre_seq_len} \ + --quantization_bit 4" + + process = subprocess.Popen(command, shell=True, cwd=ptuning_directory) + try: + process.communicate(timeout=3600*24) + except subprocess.TimeoutExpired: + process.kill() + return diff --git a/crazy_functions/crazy_functions_test.py b/crazy_functions/crazy_functions_test.py index a614aac..8b6b540 100644 --- a/crazy_functions/crazy_functions_test.py +++ b/crazy_functions/crazy_functions_test.py @@ -212,11 +212,17 @@ def test_Latex(): # cli_printer.print(cb) # print(cb) def test_chatglm_finetune(): - from crazy_functions.chatglm微调工具 import 微调数据集生成 + from crazy_functions.chatglm微调工具 import 微调数据集生成, 启动微调 txt = 'build/dev.json' plugin_kwargs = {"advanced_arg":"--llm_to_learn=gpt-3.5-turbo --prompt_prefix='根据下面的服装类型提示,想象一个穿着者,对这个人外貌、身处的环境、内心世界、人设进行描写。要求:100字以内,用第二人称。' --system_prompt=''" } - for cookies, cb, hist, msg in (微调数据集生成)(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port): + # for cookies, cb, hist, msg in (微调数据集生成)(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port): + # cli_printer.print(cb) + + plugin_kwargs = {"advanced_arg": + " --pre_seq_len=128 --learning_rate=2e-2 --num_gpus=1 --json_dataset='t_code.json' --ptuning_directory='/home/hmp/ChatGLM2-6B/ptuning' " } + + for cookies, cb, hist, msg in (启动微调)(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port): cli_printer.print(cb) diff --git a/request_llm/bridge_all.py b/request_llm/bridge_all.py index 13f49bd..c8ec2da 100644 --- a/request_llm/bridge_all.py +++ b/request_llm/bridge_all.py @@ -168,7 +168,8 @@ model_info = { } -AVAIL_LLM_MODELS, = get_conf("AVAIL_LLM_MODELS") +AVAIL_LLM_MODELS, LLM_MODEL = get_conf("AVAIL_LLM_MODELS", "LLM_MODEL") +AVAIL_LLM_MODELS = AVAIL_LLM_MODELS + [LLM_MODEL] if "jittorllms_rwkv" in AVAIL_LLM_MODELS: from .bridge_jittorllms_rwkv import predict_no_ui_long_connection as rwkv_noui from .bridge_jittorllms_rwkv import predict as rwkv_ui @@ -269,6 +270,24 @@ if "newbing" in AVAIL_LLM_MODELS: # same with newbing-free }) except: print(trimmed_format_exc()) +if "chatglmft" in AVAIL_LLM_MODELS: # same with newbing-free + try: + from .bridge_chatglmft import predict_no_ui_long_connection as chatglmft_noui + from .bridge_chatglmft import predict as chatglmft_ui + # claude + model_info.update({ + "chatglmft": { + "fn_with_ui": chatglmft_ui, + "fn_without_ui": chatglmft_noui, + "endpoint": None, + "max_token": 4096, + "tokenizer": tokenizer_gpt35, + "token_cnt": get_token_num_gpt35, + } + }) + except: + print(trimmed_format_exc()) + def LLM_CATCH_EXCEPTION(f): """ @@ -372,6 +391,6 @@ def predict(inputs, llm_kwargs, *args, **kwargs): additional_fn代表点击的哪个按钮,按钮见functional.py """ - method = model_info[llm_kwargs['llm_model']]["fn_with_ui"] + method = model_info[llm_kwargs['llm_model']]["fn_with_ui"] # 如果这里报错,检查config中的AVAIL_LLM_MODELS选项 yield from method(inputs, llm_kwargs, *args, **kwargs) diff --git a/request_llm/bridge_chatglmft.py b/request_llm/bridge_chatglmft.py new file mode 100644 index 0000000..cd0ccb9 --- /dev/null +++ b/request_llm/bridge_chatglmft.py @@ -0,0 +1,210 @@ + +from transformers import AutoModel, AutoTokenizer +import time +import os +import json +import threading +import importlib +from toolbox import update_ui, get_conf +from multiprocessing import Process, Pipe + +load_message = "ChatGLMFT尚未加载,加载需要一段时间。注意,取决于`config.py`的配置,ChatGLMFT消耗大量的内存(CPU)或显存(GPU),也许会导致低配计算机卡死 ……" + +def string_to_options(arguments): + import argparse + import shlex + # Create an argparse.ArgumentParser instance + parser = argparse.ArgumentParser() + # Add command-line arguments + parser.add_argument("--llm_to_learn", type=str, help="LLM model to learn", default="gpt-3.5-turbo") + parser.add_argument("--prompt_prefix", type=str, help="Prompt prefix", default='') + parser.add_argument("--system_prompt", type=str, help="System prompt", default='') + parser.add_argument("--batch", type=int, help="System prompt", default=50) + # Parse the arguments + args = parser.parse_args(shlex.split(arguments)) + return args + + +################################################################################# +class GetGLMFTHandle(Process): + def __init__(self): + super().__init__(daemon=True) + self.parent, self.child = Pipe() + self.chatglmft_model = None + self.chatglmft_tokenizer = None + self.info = "" + self.success = True + self.check_dependency() + self.start() + self.threadLock = threading.Lock() + + def check_dependency(self): + try: + import sentencepiece + self.info = "依赖检测通过" + self.success = True + except: + self.info = "缺少ChatGLMFT的依赖,如果要使用ChatGLMFT,除了基础的pip依赖以外,您还需要运行`pip install -r request_llm/requirements_chatglm.txt`安装ChatGLM的依赖。" + self.success = False + + def ready(self): + return self.chatglmft_model is not None + + def run(self): + # 子进程执行 + # 第一次运行,加载参数 + retry = 0 + while True: + try: + if self.chatglmft_model is None: + from transformers import AutoConfig + import torch + # conf = 'request_llm/current_ptune_model.json' + # if not os.path.exists(conf): raise RuntimeError('找不到微调模型信息') + # with open(conf, 'r', encoding='utf8') as f: + # model_args = json.loads(f.read()) + ChatGLM_PTUNING_CHECKPOINT, = get_conf('ChatGLM_PTUNING_CHECKPOINT') + assert os.path.exists(ChatGLM_PTUNING_CHECKPOINT), "找不到微调模型检查点" + conf = os.path.join(ChatGLM_PTUNING_CHECKPOINT, "config.json") + with open(conf, 'r', encoding='utf8') as f: + model_args = json.loads(f.read()) + if 'model_name_or_path' not in model_args: + model_args['model_name_or_path'] = model_args['_name_or_path'] + self.chatglmft_tokenizer = AutoTokenizer.from_pretrained( + model_args['model_name_or_path'], trust_remote_code=True) + config = AutoConfig.from_pretrained( + model_args['model_name_or_path'], trust_remote_code=True) + + config.pre_seq_len = model_args['pre_seq_len'] + config.prefix_projection = model_args['prefix_projection'] + + print(f"Loading prefix_encoder weight from {ChatGLM_PTUNING_CHECKPOINT}") + model = AutoModel.from_pretrained(model_args['model_name_or_path'], config=config, trust_remote_code=True) + prefix_state_dict = torch.load(os.path.join(ChatGLM_PTUNING_CHECKPOINT, "pytorch_model.bin")) + new_prefix_state_dict = {} + for k, v in prefix_state_dict.items(): + if k.startswith("transformer.prefix_encoder."): + new_prefix_state_dict[k[len("transformer.prefix_encoder."):]] = v + model.transformer.prefix_encoder.load_state_dict(new_prefix_state_dict) + + if model_args['quantization_bit'] is not None: + print(f"Quantized to {model_args['quantization_bit']} bit") + model = model.quantize(model_args['quantization_bit']) + model = model.cuda() + if model_args['pre_seq_len'] is not None: + # P-tuning v2 + model.transformer.prefix_encoder.float() + self.chatglmft_model = model.eval() + + break + else: + break + except Exception as e: + retry += 1 + if retry > 3: + self.child.send('[Local Message] Call ChatGLMFT fail 不能正常加载ChatGLMFT的参数。') + raise RuntimeError("不能正常加载ChatGLMFT的参数!") + + while True: + # 进入任务等待状态 + kwargs = self.child.recv() + # 收到消息,开始请求 + try: + for response, history in self.chatglmft_model.stream_chat(self.chatglmft_tokenizer, **kwargs): + self.child.send(response) + # # 中途接收可能的终止指令(如果有的话) + # if self.child.poll(): + # command = self.child.recv() + # if command == '[Terminate]': break + except: + from toolbox import trimmed_format_exc + self.child.send('[Local Message] Call ChatGLMFT fail.' + '\n```\n' + trimmed_format_exc() + '\n```\n') + # 请求处理结束,开始下一个循环 + self.child.send('[Finish]') + + def stream_chat(self, **kwargs): + # 主进程执行 + self.threadLock.acquire() + self.parent.send(kwargs) + while True: + res = self.parent.recv() + if res != '[Finish]': + yield res + else: + break + self.threadLock.release() + +global glmft_handle +glmft_handle = None +################################################################################# +def predict_no_ui_long_connection(inputs, llm_kwargs, history=[], sys_prompt="", observe_window=[], console_slience=False): + """ + 多线程方法 + 函数的说明请见 request_llm/bridge_all.py + """ + global glmft_handle + if glmft_handle is None: + glmft_handle = GetGLMFTHandle() + if len(observe_window) >= 1: observe_window[0] = load_message + "\n\n" + glmft_handle.info + if not glmft_handle.success: + error = glmft_handle.info + glmft_handle = None + raise RuntimeError(error) + + # chatglmft 没有 sys_prompt 接口,因此把prompt加入 history + history_feedin = [] + history_feedin.append(["What can I do?", sys_prompt]) + for i in range(len(history)//2): + history_feedin.append([history[2*i], history[2*i+1]] ) + + watch_dog_patience = 5 # 看门狗 (watchdog) 的耐心, 设置5秒即可 + response = "" + for response in glmft_handle.stream_chat(query=inputs, history=history_feedin, max_length=llm_kwargs['max_length'], top_p=llm_kwargs['top_p'], temperature=llm_kwargs['temperature']): + if len(observe_window) >= 1: observe_window[0] = response + if len(observe_window) >= 2: + if (time.time()-observe_window[1]) > watch_dog_patience: + raise RuntimeError("程序终止。") + return response + + + +def predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_prompt='', stream = True, additional_fn=None): + """ + 单线程方法 + 函数的说明请见 request_llm/bridge_all.py + """ + chatbot.append((inputs, "")) + + global glmft_handle + if glmft_handle is None: + glmft_handle = GetGLMFTHandle() + chatbot[-1] = (inputs, load_message + "\n\n" + glmft_handle.info) + yield from update_ui(chatbot=chatbot, history=[]) + if not glmft_handle.success: + glmft_handle = None + return + + if additional_fn is not None: + import core_functional + importlib.reload(core_functional) # 热更新prompt + core_functional = core_functional.get_core_functions() + if "PreProcess" in core_functional[additional_fn]: inputs = core_functional[additional_fn]["PreProcess"](inputs) # 获取预处理函数(如果有的话) + inputs = core_functional[additional_fn]["Prefix"] + inputs + core_functional[additional_fn]["Suffix"] + + # 处理历史信息 + history_feedin = [] + history_feedin.append(["What can I do?", system_prompt] ) + for i in range(len(history)//2): + history_feedin.append([history[2*i], history[2*i+1]] ) + + # 开始接收chatglmft的回复 + response = "[Local Message]: 等待ChatGLMFT响应中 ..." + for response in glmft_handle.stream_chat(query=inputs, history=history_feedin, max_length=llm_kwargs['max_length'], top_p=llm_kwargs['top_p'], temperature=llm_kwargs['temperature']): + chatbot[-1] = (inputs, response) + yield from update_ui(chatbot=chatbot, history=history) + + # 总结输出 + if response == "[Local Message]: 等待ChatGLMFT响应中 ...": + response = "[Local Message]: ChatGLMFT响应异常 ..." + history.extend([inputs, response]) + yield from update_ui(chatbot=chatbot, history=history) diff --git a/version b/version index 0a59059..36fd151 100644 --- a/version +++ b/version @@ -1,5 +1,5 @@ { - "version": 3.44, + "version": 3.45, "show_feature": true, - "new_feature": "[改善UI] 动态ChatBot窗口高度 <-> 修复Azure接口的BUG <-> 完善多语言模块 <-> 完善本地Latex矫错和翻译功能 <-> 增加gpt-3.5-16k的支持 <-> 新增最强Arxiv论文翻译插件 <-> 修复gradio复制按钮BUG <-> 修复PDF翻译的BUG, 新增HTML中英双栏对照 <-> 添加了OpenAI图片生成插件" + "new_feature": "支持加载自定义的ChatGLM2微调模型 <-> [改善UI] 动态ChatBot窗口高度 <-> 修复Azure接口的BUG <-> 完善多语言模块 <-> 完善本地Latex矫错和翻译功能 <-> 增加gpt-3.5-16k的支持 <-> 新增最强Arxiv论文翻译插件 <-> 修复gradio复制按钮BUG <-> 修复PDF翻译的BUG, 新增HTML中英双栏对照 <-> 添加了OpenAI图片生成插件" }