Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0aeb5b28cd | ||
|
|
1dd1720d38 | ||
|
|
19be0490af | ||
|
|
0c9e18291a | ||
|
|
9f47d0f714 | ||
|
|
7a254c150f | ||
|
|
3648648b3d | ||
|
|
1da60b7a0c | ||
|
|
c40f6f00bb | ||
|
|
a239abac50 | ||
|
|
1042d28e1f | ||
|
|
7b75422c26 | ||
|
|
99817e9040 | ||
|
|
c9fa26405d | ||
|
|
005232afa6 | ||
|
|
5b8cc5a899 | ||
|
|
a4137e7170 | ||
|
|
aaf44750d9 | ||
|
|
bd6eb90449 | ||
|
|
b5a48369a4 | ||
|
|
6b5bdbe98a | ||
|
|
69624c66d7 | ||
|
|
69be335d22 | ||
|
|
bb1e410cb4 | ||
|
|
9ccc53fa96 | ||
|
|
4b83486b3d | ||
|
|
417c8325de | ||
|
|
d51ae6abb2 | ||
|
|
fff7b8ef91 | ||
|
|
c59eb8ff9e | ||
|
|
a74f0a9343 | ||
|
|
f4905a60e2 | ||
|
|
a562849e4c | ||
|
|
6105b7f73b | ||
|
|
3486fb5c10 | ||
|
|
5f7a1a3da3 | ||
|
|
8d086ce7c0 | ||
|
|
b188c4a2b5 | ||
|
|
10cf456aa8 | ||
|
|
4043db7f33 | ||
|
|
9a192fd473 | ||
|
|
9f91fca4d2 | ||
|
|
160b001bef | ||
|
|
1d912bc10d | ||
|
|
51b3f8adca | ||
|
|
16de1812d3 | ||
|
|
641b96548a | ||
|
|
34f4ba211d | ||
|
|
19aba350a3 | ||
|
|
ab57f4bfb0 | ||
|
|
b7e0a48cd2 | ||
|
|
58b051ead3 | ||
|
|
0c7378e096 | ||
|
|
a52ae14457 | ||
|
|
ce3e9b6289 | ||
|
|
9f07531a16 | ||
|
|
e35f7a7186 | ||
|
|
7c91cfebfa | ||
|
|
12c36a68ce | ||
|
|
998e127b2f |
@@ -0,0 +1,19 @@
|
|||||||
|
---
|
||||||
|
name: Bug report
|
||||||
|
about: Create a report to help us improve
|
||||||
|
title: ''
|
||||||
|
labels: ''
|
||||||
|
assignees: ''
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Describe the bug 简述**
|
||||||
|
|
||||||
|
**Screen Shot 截图**
|
||||||
|
|
||||||
|
**Terminal Traceback 终端traceback(如果有)**
|
||||||
|
|
||||||
|
|
||||||
|
Before submitting an issue 提交issue之前:
|
||||||
|
- Please try to upgrade your code. 如果您的代码不是最新的,建议您先尝试更新代码
|
||||||
|
- Please check project wiki for common problem solutions.项目[wiki](https://github.com/binary-husky/chatgpt_academic/wiki)有一些常见问题的解决方法
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
---
|
||||||
|
name: Feature request
|
||||||
|
about: Suggest an idea for this project
|
||||||
|
title: ''
|
||||||
|
labels: ''
|
||||||
|
assignees: ''
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
+2
-1
@@ -139,4 +139,5 @@ config_private.py
|
|||||||
gpt_log
|
gpt_log
|
||||||
private.md
|
private.md
|
||||||
private_upload
|
private_upload
|
||||||
other_llms
|
other_llms
|
||||||
|
cradle.py
|
||||||
@@ -2,9 +2,9 @@
|
|||||||
|
|
||||||
# ChatGPT 学术优化
|
# ChatGPT 学术优化
|
||||||
|
|
||||||
**如果喜欢这个项目,请给它一个Star;如果你发明了更好用的学术快捷键,欢迎发issue或者pull requests(dev分支)**
|
**如果喜欢这个项目,请给它一个Star;如果你发明了更好用的快捷键或函数插件,欢迎发issue或者pull requests(dev分支)**
|
||||||
|
|
||||||
If you like this project, please give it a Star. If you've come up with more useful academic shortcuts, feel free to open an issue or pull request (to `dev` branch).
|
If you like this project, please give it a Star. If you've come up with more useful academic shortcuts or functional plugins, feel free to open an issue or pull request (to `dev` branch).
|
||||||
|
|
||||||
```
|
```
|
||||||
代码中参考了很多其他优秀项目中的设计,主要包括:
|
代码中参考了很多其他优秀项目中的设计,主要包括:
|
||||||
@@ -20,11 +20,11 @@ https://github.com/polarwinkel/mdtex2html
|
|||||||
|
|
||||||
> **Note**
|
> **Note**
|
||||||
>
|
>
|
||||||
> 1.请注意只有“红颜色”标识的函数插件(按钮)才支持读取文件。目前暂不能完善地支持pdf/word格式文献的翻译解读,相关函数函件正在测试中。
|
> 1.请注意只有“红颜色”标识的函数插件(按钮)才支持读取文件。目前对pdf/word格式文件的支持插件正在逐步完善中,需要更多developer的帮助。
|
||||||
>
|
>
|
||||||
> 2.本项目中每个文件的功能都在自译解[`project_self_analysis.md`](https://github.com/binary-husky/chatgpt_academic/wiki/chatgpt-academic%E9%A1%B9%E7%9B%AE%E8%87%AA%E8%AF%91%E8%A7%A3%E6%8A%A5%E5%91%8A)详细说明。随着版本的迭代,您也可以随时自行点击相关函数插件,调用GPT重新生成项目的自我解析报告。常见问题汇总在[`wiki`](https://github.com/binary-husky/chatgpt_academic/wiki/%E5%B8%B8%E8%A7%81%E9%97%AE%E9%A2%98)当中。
|
> 2.本项目中每个文件的功能都在自译解[`self_analysis.md`](https://github.com/binary-husky/chatgpt_academic/wiki/chatgpt-academic%E9%A1%B9%E7%9B%AE%E8%87%AA%E8%AF%91%E8%A7%A3%E6%8A%A5%E5%91%8A)详细说明。随着版本的迭代,您也可以随时自行点击相关函数插件,调用GPT重新生成项目的自我解析报告。常见问题汇总在[`wiki`](https://github.com/binary-husky/chatgpt_academic/wiki/%E5%B8%B8%E8%A7%81%E9%97%AE%E9%A2%98)当中。
|
||||||
>
|
>
|
||||||
> 3.如果您不太习惯部分中文命名的函数,您可以随时点击相关函数插件,调用GPT一键生成纯英文的项目源代码。
|
> 3.如果您不太习惯部分中文命名的函数、注释或者界面,您可以随时点击相关函数插件,调用ChatGPT一键生成纯英文的项目源代码。
|
||||||
|
|
||||||
<div align="center">
|
<div align="center">
|
||||||
|
|
||||||
@@ -35,17 +35,19 @@ https://github.com/polarwinkel/mdtex2html
|
|||||||
一键代码解释 | 可以正确显示代码、解释代码
|
一键代码解释 | 可以正确显示代码、解释代码
|
||||||
自定义快捷键 | 支持自定义快捷键
|
自定义快捷键 | 支持自定义快捷键
|
||||||
配置代理服务器 | 支持配置代理服务器
|
配置代理服务器 | 支持配置代理服务器
|
||||||
模块化设计 | 支持自定义高阶的实验性功能
|
模块化设计 | 支持自定义高阶的实验性功能与[函数插件],插件支持[热更新](https://github.com/binary-husky/chatgpt_academic/wiki/%E5%87%BD%E6%95%B0%E6%8F%92%E4%BB%B6%E6%8C%87%E5%8D%97)
|
||||||
自我程序剖析 | [函数插件] 一键读懂本项目的源代码
|
自我程序剖析 | [函数插件] 一键读懂本项目的源代码
|
||||||
程序剖析 | [函数插件] 一键可以剖析其他Python/C++等项目
|
程序剖析 | [函数插件] 一键可以剖析其他Python/C/C++/Java项目树
|
||||||
读论文 | [函数插件] 一键解读latex论文全文并生成摘要
|
读论文 | [函数插件] 一键解读latex论文全文并生成摘要
|
||||||
arxiv小助手 | [函数插件] 输入url一键翻译摘要+下载论文
|
|
||||||
批量注释生成 | [函数插件] 一键批量生成函数注释
|
批量注释生成 | [函数插件] 一键批量生成函数注释
|
||||||
chat分析报告生成 | [函数插件] 运行后自动生成总结汇报
|
chat分析报告生成 | [函数插件] 运行后自动生成总结汇报
|
||||||
|
arxiv小助手 | [函数插件] 输入arxiv文章url即可一键翻译摘要+下载PDF
|
||||||
公式显示 | 可以同时显示公式的tex形式和渲染形式
|
公式显示 | 可以同时显示公式的tex形式和渲染形式
|
||||||
图片显示 | 可以在markdown中显示图片
|
图片显示 | 可以在markdown中显示图片
|
||||||
|
多线程函数插件支持 | 支持多线调用chatgpt,一键处理海量文本或程序
|
||||||
支持GPT输出的markdown表格 | 可以输出支持GPT的markdown表格
|
支持GPT输出的markdown表格 | 可以输出支持GPT的markdown表格
|
||||||
本地大语言模型接口 | 借助[TGUI](https://github.com/oobabooga/text-generation-webui)接入galactica等本地语言模型
|
启动暗色gradio[主题](https://github.com/binary-husky/chatgpt_academic/issues/173) | 在浏览器url后面添加```/?__dark-theme=true```可以切换dark主题
|
||||||
|
huggingface免科学上网[在线体验](https://huggingface.co/spaces/qingxu98/gpt-academic) | 登陆huggingface后复制[此空间](https://huggingface.co/spaces/qingxu98/gpt-academic)
|
||||||
…… | ……
|
…… | ……
|
||||||
|
|
||||||
</div>
|
</div>
|
||||||
@@ -113,7 +115,7 @@ python -m pip install -r requirements.txt
|
|||||||
# (选择二.2)conda activate gptac_venv
|
# (选择二.2)conda activate gptac_venv
|
||||||
# (选择二.3)python -m pip install -r requirements.txt
|
# (选择二.3)python -m pip install -r requirements.txt
|
||||||
|
|
||||||
# 备注:使用官方pip源或者阿里pip源,其他pip源(如清华pip)有可能出问题,临时换源方法:
|
# 备注:使用官方pip源或者阿里pip源,其他pip源(如一些大学的pip)有可能出问题,临时换源方法:
|
||||||
# python -m pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
# python -m pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -194,7 +196,7 @@ input区域 输入 ./crazy_functions/test_project/python/dqn , 然后点击 "[
|
|||||||
如果你发明了更好用的学术快捷键,欢迎发issue或者pull requests!
|
如果你发明了更好用的学术快捷键,欢迎发issue或者pull requests!
|
||||||
|
|
||||||
## 配置代理
|
## 配置代理
|
||||||
|
### 方法一:常规方法
|
||||||
在```config.py```中修改端口与代理软件对应
|
在```config.py```中修改端口与代理软件对应
|
||||||
|
|
||||||
<div align="center">
|
<div align="center">
|
||||||
@@ -206,6 +208,8 @@ input区域 输入 ./crazy_functions/test_project/python/dqn , 然后点击 "[
|
|||||||
```
|
```
|
||||||
python check_proxy.py
|
python check_proxy.py
|
||||||
```
|
```
|
||||||
|
### 方法二:纯新手教程
|
||||||
|
[纯新手教程](https://github.com/binary-husky/chatgpt_academic/wiki/%E4%BB%A3%E7%90%86%E8%BD%AF%E4%BB%B6%E9%97%AE%E9%A2%98%E7%9A%84%E6%96%B0%E6%89%8B%E8%A7%A3%E5%86%B3%E6%96%B9%E6%B3%95%EF%BC%88%E6%96%B9%E6%B3%95%E5%8F%AA%E9%80%82%E7%94%A8%E4%BA%8E%E6%96%B0%E6%89%8B%EF%BC%89)
|
||||||
|
|
||||||
## 兼容性测试
|
## 兼容性测试
|
||||||
|
|
||||||
@@ -249,13 +253,28 @@ python check_proxy.py
|
|||||||
|
|
||||||
### 模块化功能设计
|
### 模块化功能设计
|
||||||
<div align="center">
|
<div align="center">
|
||||||
<img src="https://user-images.githubusercontent.com/96192199/227504981-4c6c39c0-ae79-47e6-bffe-0e6442d9da65.png" height="400" >
|
<img src="https://user-images.githubusercontent.com/96192199/229288270-093643c1-0018-487a-81e6-1d7809b6e90f.png" height="400" >
|
||||||
<img src="https://user-images.githubusercontent.com/96192199/227504931-19955f78-45cd-4d1c-adac-e71e50957915.png" height="400" >
|
<img src="https://user-images.githubusercontent.com/96192199/227504931-19955f78-45cd-4d1c-adac-e71e50957915.png" height="400" >
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
## Todo:
|
|
||||||
|
|
||||||
- (Top Priority) 调用另一个开源项目text-generation-webui的web接口,使用其他llm模型
|
### 源代码转译英文
|
||||||
- 总结大工程源代码时,文本过长、token溢出的问题(目前的方法是直接二分丢弃处理溢出,过于粗暴,有效信息大量丢失)
|
|
||||||
- UI不够美观
|
|
||||||
|
|
||||||
|
<div align="center">
|
||||||
|
<img src="https://user-images.githubusercontent.com/96192199/229720562-fe6c3508-6142-4635-a83d-21eb3669baee.png" height="400" >
|
||||||
|
</div>
|
||||||
|
|
||||||
|
## Todo 与 版本规划:
|
||||||
|
|
||||||
|
- version 3 (Todo):
|
||||||
|
- - 支持gpt4和其他更多llm
|
||||||
|
- version 2.3+ (Todo):
|
||||||
|
- - 总结大工程源代码时文本过长、token溢出的问题
|
||||||
|
- - 实现项目打包部署
|
||||||
|
- - 函数插件参数接口优化
|
||||||
|
- - 自更新
|
||||||
|
- version 2.3: 增强多线程交互性
|
||||||
|
- version 2.2: 函数插件支持热重载
|
||||||
|
- version 2.1: 可折叠式布局
|
||||||
|
- version 2.0: 引入模块化函数插件
|
||||||
|
- version 1.0: 基础功能
|
||||||
@@ -19,6 +19,30 @@ def check_proxy(proxies):
|
|||||||
return result
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def auto_update():
|
||||||
|
from toolbox import get_conf
|
||||||
|
import requests, time, json
|
||||||
|
proxies, = get_conf('proxies')
|
||||||
|
response = requests.get("https://raw.githubusercontent.com/binary-husky/chatgpt_academic/master/version",
|
||||||
|
proxies=proxies, timeout=1)
|
||||||
|
remote_json_data = json.loads(response.text)
|
||||||
|
remote_version = remote_json_data['version']
|
||||||
|
if remote_json_data["show_feature"]:
|
||||||
|
new_feature = "新功能:" + remote_json_data["new_feature"]
|
||||||
|
else:
|
||||||
|
new_feature = ""
|
||||||
|
with open('./version', 'r', encoding='utf8') as f:
|
||||||
|
current_version = f.read()
|
||||||
|
current_version = json.loads(current_version)['version']
|
||||||
|
if (remote_version - current_version) >= 0.05:
|
||||||
|
print(f'\n新版本可用。新版本:{remote_version},当前版本:{current_version}。{new_feature}')
|
||||||
|
print('Github更新地址:\nhttps://github.com/binary-husky/chatgpt_academic\n')
|
||||||
|
time.sleep(3)
|
||||||
|
return
|
||||||
|
else:
|
||||||
|
return
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
if __name__ == '__main__':
|
||||||
import os; os.environ['no_proxy'] = '*' # 避免代理网络产生意外污染
|
import os; os.environ['no_proxy'] = '*' # 避免代理网络产生意外污染
|
||||||
from toolbox import get_conf
|
from toolbox import get_conf
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
# [step 1]>> 例如: API_KEY = "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" (此key无效)
|
# [step 1]>> 例如: API_KEY = "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" (此key无效)
|
||||||
API_KEY = "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"
|
API_KEY = "sk-此处填API密钥"
|
||||||
|
|
||||||
# [step 2]>> 改为True应用代理,如果直接在海外服务器部署,此处不修改
|
# [step 2]>> 改为True应用代理,如果直接在海外服务器部署,此处不修改
|
||||||
USE_PROXY = False
|
USE_PROXY = False
|
||||||
@@ -22,7 +22,10 @@ else:
|
|||||||
|
|
||||||
# [step 3]>> 以下配置可以优化体验,但大部分场合下并不需要修改
|
# [step 3]>> 以下配置可以优化体验,但大部分场合下并不需要修改
|
||||||
# 对话窗的高度
|
# 对话窗的高度
|
||||||
CHATBOT_HEIGHT = 1116
|
CHATBOT_HEIGHT = 1115
|
||||||
|
|
||||||
|
# 窗口布局
|
||||||
|
LAYOUT = "LEFT-RIGHT" # "LEFT-RIGHT"(左右布局) # "TOP-DOWN"(上下布局)
|
||||||
|
|
||||||
# 发送请求到OpenAI后,等待多久判定为超时
|
# 发送请求到OpenAI后,等待多久判定为超时
|
||||||
TIMEOUT_SECONDS = 25
|
TIMEOUT_SECONDS = 25
|
||||||
@@ -34,7 +37,7 @@ WEB_PORT = -1
|
|||||||
MAX_RETRY = 2
|
MAX_RETRY = 2
|
||||||
|
|
||||||
# OpenAI模型选择是(gpt4现在只对申请成功的人开放)
|
# OpenAI模型选择是(gpt4现在只对申请成功的人开放)
|
||||||
LLM_MODEL = "TGUI:galactica-1.3b@localhost:7860" # "gpt-3.5-turbo"
|
LLM_MODEL = "gpt-3.5-turbo"
|
||||||
|
|
||||||
# OpenAI的API_URL
|
# OpenAI的API_URL
|
||||||
API_URL = "https://api.openai.com/v1/chat/completions"
|
API_URL = "https://api.openai.com/v1/chat/completions"
|
||||||
@@ -42,5 +45,5 @@ API_URL = "https://api.openai.com/v1/chat/completions"
|
|||||||
# 设置并行使用的线程数
|
# 设置并行使用的线程数
|
||||||
CONCURRENT_COUNT = 100
|
CONCURRENT_COUNT = 100
|
||||||
|
|
||||||
# 设置用户名和密码
|
# 设置用户名和密码(相关功能不稳定,与gradio版本和网络都相关,如果本地使用不建议加这个)
|
||||||
AUTHENTICATION = [] # [("username", "password"), ("username2", "password2"), ...]
|
AUTHENTICATION = [] # [("username", "password"), ("username2", "password2"), ...]
|
||||||
|
|||||||
@@ -134,8 +134,7 @@ def get_name(_url_):
|
|||||||
@CatchException
|
@CatchException
|
||||||
def 下载arxiv论文并翻译摘要(txt, top_p, temperature, chatbot, history, systemPromptTxt, WEB_PORT):
|
def 下载arxiv论文并翻译摘要(txt, top_p, temperature, chatbot, history, systemPromptTxt, WEB_PORT):
|
||||||
|
|
||||||
CRAZY_FUNCTION_INFO = "下载arxiv论文并翻译摘要,作者 binary-husky。正在提取摘要并下载PDF文档……"
|
CRAZY_FUNCTION_INFO = "下载arxiv论文并翻译摘要,函数插件作者[binary-husky]。正在提取摘要并下载PDF文档……"
|
||||||
raise RuntimeError()
|
|
||||||
import glob
|
import glob
|
||||||
import os
|
import os
|
||||||
|
|
||||||
@@ -180,8 +179,8 @@ def 下载arxiv论文并翻译摘要(txt, top_p, temperature, chatbot, history,
|
|||||||
# 写入文件
|
# 写入文件
|
||||||
import shutil
|
import shutil
|
||||||
# 重置文件的创建时间
|
# 重置文件的创建时间
|
||||||
shutil.copyfile(pdf_path, pdf_path.replace('.pdf', '.autodownload.pdf')); os.remove(pdf_path)
|
shutil.copyfile(pdf_path, f'./gpt_log/{os.path.basename(pdf_path)}'); os.remove(pdf_path)
|
||||||
res = write_results_to_file(history)
|
res = write_results_to_file(history)
|
||||||
chatbot.append(("完成了吗?", res))
|
chatbot.append(("完成了吗?", res + "\n\nPDF文件也已经下载"))
|
||||||
yield chatbot, history, msg
|
yield chatbot, history, msg
|
||||||
|
|
||||||
|
|||||||
+111
-21
@@ -1,41 +1,123 @@
|
|||||||
import threading
|
import threading
|
||||||
from predict import predict_no_ui_long_connection
|
from predict import predict_no_ui_long_connection
|
||||||
from toolbox import CatchException, write_results_to_file
|
from toolbox import CatchException, write_results_to_file, report_execption
|
||||||
|
|
||||||
|
def extract_code_block_carefully(txt):
|
||||||
|
splitted = txt.split('```')
|
||||||
|
n_code_block_seg = len(splitted) - 1
|
||||||
|
if n_code_block_seg <= 1: return txt
|
||||||
|
# 剩下的情况都开头除去 ``` 结尾除去一次 ```
|
||||||
|
txt_out = '```'.join(splitted[1:-1])
|
||||||
|
return txt_out
|
||||||
|
|
||||||
|
def breakdown_txt_to_satisfy_token_limit(txt, get_token_fn, limit, must_break_at_empty_line=True):
|
||||||
|
def cut(txt_tocut, must_break_at_empty_line): # 递归
|
||||||
|
if get_token_fn(txt_tocut) <= limit:
|
||||||
|
return [txt_tocut]
|
||||||
|
else:
|
||||||
|
lines = txt_tocut.split('\n')
|
||||||
|
estimated_line_cut = limit / get_token_fn(txt_tocut) * len(lines)
|
||||||
|
estimated_line_cut = int(estimated_line_cut)
|
||||||
|
for cnt in reversed(range(estimated_line_cut)):
|
||||||
|
if must_break_at_empty_line:
|
||||||
|
if lines[cnt] != "": continue
|
||||||
|
print(cnt)
|
||||||
|
prev = "\n".join(lines[:cnt])
|
||||||
|
post = "\n".join(lines[cnt:])
|
||||||
|
if get_token_fn(prev) < limit: break
|
||||||
|
if cnt == 0:
|
||||||
|
print('what the f?')
|
||||||
|
raise RuntimeError("存在一行极长的文本!")
|
||||||
|
print(len(post))
|
||||||
|
# 列表递归接龙
|
||||||
|
result = [prev]
|
||||||
|
result.extend(cut(post, must_break_at_empty_line))
|
||||||
|
return result
|
||||||
|
try:
|
||||||
|
return cut(txt, must_break_at_empty_line=True)
|
||||||
|
except RuntimeError:
|
||||||
|
return cut(txt, must_break_at_empty_line=False)
|
||||||
|
|
||||||
|
|
||||||
|
def break_txt_into_half_at_some_linebreak(txt):
|
||||||
|
lines = txt.split('\n')
|
||||||
|
n_lines = len(lines)
|
||||||
|
pre = lines[:(n_lines//2)]
|
||||||
|
post = lines[(n_lines//2):]
|
||||||
|
return "\n".join(pre), "\n".join(post)
|
||||||
|
|
||||||
|
|
||||||
@CatchException
|
@CatchException
|
||||||
def 全项目切换英文(txt, top_p, temperature, chatbot, history, sys_prompt, WEB_PORT):
|
def 全项目切换英文(txt, top_p, temperature, chatbot, history, sys_prompt, WEB_PORT):
|
||||||
history = [] # 清空历史,以免输入溢出
|
# 第1步:清空历史,以免输入溢出
|
||||||
# 集合文件
|
history = []
|
||||||
import time, glob, os
|
|
||||||
|
# 第2步:尝试导入依赖,如果缺少依赖,则给出安装建议
|
||||||
|
try:
|
||||||
|
import openai, transformers
|
||||||
|
except:
|
||||||
|
report_execption(chatbot, history,
|
||||||
|
a = f"解析项目: {txt}",
|
||||||
|
b = f"导入软件依赖失败。使用该模块需要额外依赖,安装方法```pip install --upgrade openai transformers```。")
|
||||||
|
yield chatbot, history, '正常'
|
||||||
|
return
|
||||||
|
|
||||||
|
# 第3步:集合文件
|
||||||
|
import time, glob, os, shutil, re, openai
|
||||||
os.makedirs('gpt_log/generated_english_version', exist_ok=True)
|
os.makedirs('gpt_log/generated_english_version', exist_ok=True)
|
||||||
os.makedirs('gpt_log/generated_english_version/crazy_functions', exist_ok=True)
|
os.makedirs('gpt_log/generated_english_version/crazy_functions', exist_ok=True)
|
||||||
file_manifest = [f for f in glob.glob('./*.py') if ('test_project' not in f) and ('gpt_log' not in f)] + \
|
file_manifest = [f for f in glob.glob('./*.py') if ('test_project' not in f) and ('gpt_log' not in f)] + \
|
||||||
[f for f in glob.glob('./crazy_functions/*.py') if ('test_project' not in f) and ('gpt_log' not in f)]
|
[f for f in glob.glob('./crazy_functions/*.py') if ('test_project' not in f) and ('gpt_log' not in f)]
|
||||||
|
# file_manifest = ['./toolbox.py']
|
||||||
i_say_show_user_buffer = []
|
i_say_show_user_buffer = []
|
||||||
|
|
||||||
# 随便显示点什么防止卡顿的感觉
|
# 第4步:随便显示点什么防止卡顿的感觉
|
||||||
for index, fp in enumerate(file_manifest):
|
for index, fp in enumerate(file_manifest):
|
||||||
# if 'test_project' in fp: continue
|
# if 'test_project' in fp: continue
|
||||||
with open(fp, 'r', encoding='utf-8') as f:
|
with open(fp, 'r', encoding='utf-8') as f:
|
||||||
file_content = f.read()
|
file_content = f.read()
|
||||||
i_say_show_user =f'[{index}/{len(file_manifest)}] 接下来请将以下代码中包含的所有中文转化为英文,只输出代码: {os.path.abspath(fp)}'
|
i_say_show_user =f'[{index}/{len(file_manifest)}] 接下来请将以下代码中包含的所有中文转化为英文,只输出转化后的英文代码,请用代码块输出代码: {os.path.abspath(fp)}'
|
||||||
i_say_show_user_buffer.append(i_say_show_user)
|
i_say_show_user_buffer.append(i_say_show_user)
|
||||||
chatbot.append((i_say_show_user, "[Local Message] 等待多线程操作,中间过程不予显示."))
|
chatbot.append((i_say_show_user, "[Local Message] 等待多线程操作,中间过程不予显示."))
|
||||||
yield chatbot, history, '正常'
|
yield chatbot, history, '正常'
|
||||||
|
|
||||||
# 任务函数
|
|
||||||
|
# 第5步:Token限制下的截断与处理
|
||||||
|
MAX_TOKEN = 3000
|
||||||
|
from transformers import GPT2TokenizerFast
|
||||||
|
print('加载tokenizer中')
|
||||||
|
tokenizer = GPT2TokenizerFast.from_pretrained("gpt2")
|
||||||
|
get_token_fn = lambda txt: len(tokenizer(txt)["input_ids"])
|
||||||
|
print('加载tokenizer结束')
|
||||||
|
|
||||||
|
|
||||||
|
# 第6步:任务函数
|
||||||
mutable_return = [None for _ in file_manifest]
|
mutable_return = [None for _ in file_manifest]
|
||||||
|
observe_window = [[""] for _ in file_manifest]
|
||||||
def thread_worker(fp,index):
|
def thread_worker(fp,index):
|
||||||
|
if index > 10:
|
||||||
|
time.sleep(60)
|
||||||
|
print('Openai 限制免费用户每分钟20次请求,降低请求频率中。')
|
||||||
with open(fp, 'r', encoding='utf-8') as f:
|
with open(fp, 'r', encoding='utf-8') as f:
|
||||||
file_content = f.read()
|
file_content = f.read()
|
||||||
i_say = f'接下来请将以下代码中包含的所有中文转化为英文,只输出代码,文件名是{fp},文件代码是 ```{file_content}```'
|
i_say_template = lambda fp, file_content: f'接下来请将以下代码中包含的所有中文转化为英文,只输出代码,文件名是{fp},文件代码是 ```{file_content}```'
|
||||||
# ** gpt request **
|
try:
|
||||||
gpt_say = predict_no_ui_long_connection(inputs=i_say, top_p=top_p, temperature=temperature, history=history, sys_prompt=sys_prompt)
|
gpt_say = ""
|
||||||
mutable_return[index] = gpt_say
|
# 分解代码文件
|
||||||
|
file_content_breakdown = breakdown_txt_to_satisfy_token_limit(file_content, get_token_fn, MAX_TOKEN)
|
||||||
|
for file_content_partial in file_content_breakdown:
|
||||||
|
i_say = i_say_template(fp, file_content_partial)
|
||||||
|
# # ** gpt request **
|
||||||
|
gpt_say_partial = predict_no_ui_long_connection(inputs=i_say, top_p=top_p, temperature=temperature, history=[], sys_prompt=sys_prompt, observe_window=observe_window[index])
|
||||||
|
gpt_say_partial = extract_code_block_carefully(gpt_say_partial)
|
||||||
|
gpt_say += gpt_say_partial
|
||||||
|
mutable_return[index] = gpt_say
|
||||||
|
except ConnectionAbortedError as token_exceed_err:
|
||||||
|
print('至少一个线程任务Token溢出而失败', e)
|
||||||
|
except Exception as e:
|
||||||
|
print('至少一个线程任务意外失败', e)
|
||||||
|
|
||||||
# 所有线程同时开始执行任务函数
|
# 第7步:所有线程同时开始执行任务函数
|
||||||
handles = [threading.Thread(target=thread_worker, args=(fp,index)) for index, fp in enumerate(file_manifest)]
|
handles = [threading.Thread(target=thread_worker, args=(fp,index)) for index, fp in enumerate(file_manifest)]
|
||||||
for h in handles:
|
for h in handles:
|
||||||
h.daemon = True
|
h.daemon = True
|
||||||
@@ -43,19 +125,23 @@ def 全项目切换英文(txt, top_p, temperature, chatbot, history, sys_prompt,
|
|||||||
chatbot.append(('开始了吗?', f'多线程操作已经开始'))
|
chatbot.append(('开始了吗?', f'多线程操作已经开始'))
|
||||||
yield chatbot, history, '正常'
|
yield chatbot, history, '正常'
|
||||||
|
|
||||||
# 循环轮询各个线程是否执行完毕
|
# 第8步:循环轮询各个线程是否执行完毕
|
||||||
cnt = 0
|
cnt = 0
|
||||||
while True:
|
while True:
|
||||||
time.sleep(1)
|
cnt += 1
|
||||||
|
time.sleep(0.2)
|
||||||
th_alive = [h.is_alive() for h in handles]
|
th_alive = [h.is_alive() for h in handles]
|
||||||
if not any(th_alive): break
|
if not any(th_alive): break
|
||||||
stat = ['执行中' if alive else '已完成' for alive in th_alive]
|
# 更好的UI视觉效果
|
||||||
stat_str = '|'.join(stat)
|
observe_win = []
|
||||||
cnt += 1
|
for thread_index, alive in enumerate(th_alive):
|
||||||
chatbot[-1] = (chatbot[-1][0], f'多线程操作已经开始,完成情况: {stat_str}' + ''.join(['.']*(cnt%4)))
|
observe_win.append("[ ..."+observe_window[thread_index][0][-60:].replace('\n','').replace('```','...').replace(' ','.').replace('<br/>','.....').replace('$','.')+"... ]")
|
||||||
|
stat = [f'执行中: {obs}\n\n' if alive else '已完成\n\n' for alive, obs in zip(th_alive, observe_win)]
|
||||||
|
stat_str = ''.join(stat)
|
||||||
|
chatbot[-1] = (chatbot[-1][0], f'多线程操作已经开始,完成情况: \n\n{stat_str}' + ''.join(['.']*(cnt%10+1)))
|
||||||
yield chatbot, history, '正常'
|
yield chatbot, history, '正常'
|
||||||
|
|
||||||
# 把结果写入文件
|
# 第9步:把结果写入文件
|
||||||
for index, h in enumerate(handles):
|
for index, h in enumerate(handles):
|
||||||
h.join() # 这里其实不需要join了,肯定已经都结束了
|
h.join() # 这里其实不需要join了,肯定已经都结束了
|
||||||
fp = file_manifest[index]
|
fp = file_manifest[index]
|
||||||
@@ -63,13 +149,17 @@ def 全项目切换英文(txt, top_p, temperature, chatbot, history, sys_prompt,
|
|||||||
i_say_show_user = i_say_show_user_buffer[index]
|
i_say_show_user = i_say_show_user_buffer[index]
|
||||||
|
|
||||||
where_to_relocate = f'gpt_log/generated_english_version/{fp}'
|
where_to_relocate = f'gpt_log/generated_english_version/{fp}'
|
||||||
with open(where_to_relocate, 'w+', encoding='utf-8') as f: f.write(gpt_say.lstrip('```').rstrip('```'))
|
if gpt_say is not None:
|
||||||
|
with open(where_to_relocate, 'w+', encoding='utf-8') as f:
|
||||||
|
f.write(gpt_say)
|
||||||
|
else: # 失败
|
||||||
|
shutil.copyfile(file_manifest[index], where_to_relocate)
|
||||||
chatbot.append((i_say_show_user, f'[Local Message] 已完成{os.path.abspath(fp)}的转化,\n\n存入{os.path.abspath(where_to_relocate)}'))
|
chatbot.append((i_say_show_user, f'[Local Message] 已完成{os.path.abspath(fp)}的转化,\n\n存入{os.path.abspath(where_to_relocate)}'))
|
||||||
history.append(i_say_show_user); history.append(gpt_say)
|
history.append(i_say_show_user); history.append(gpt_say)
|
||||||
yield chatbot, history, '正常'
|
yield chatbot, history, '正常'
|
||||||
time.sleep(1)
|
time.sleep(1)
|
||||||
|
|
||||||
# 备份一个文件
|
# 第10步:备份一个文件
|
||||||
res = write_results_to_file(history)
|
res = write_results_to_file(history)
|
||||||
chatbot.append(("生成一份任务执行报告", res))
|
chatbot.append(("生成一份任务执行报告", res))
|
||||||
yield chatbot, history, '正常'
|
yield chatbot, history, '正常'
|
||||||
|
|||||||
@@ -148,20 +148,66 @@ def 解析一个C项目(txt, top_p, temperature, chatbot, history, systemPromptT
|
|||||||
return
|
return
|
||||||
yield from 解析源代码(file_manifest, project_folder, top_p, temperature, chatbot, history, systemPromptTxt)
|
yield from 解析源代码(file_manifest, project_folder, top_p, temperature, chatbot, history, systemPromptTxt)
|
||||||
|
|
||||||
|
|
||||||
@CatchException
|
@CatchException
|
||||||
def 解析一个Golang项目(txt, top_p, temperature, chatbot, history, systemPromptTxt, WEB_PORT):
|
def 解析一个Java项目(txt, top_p, temperature, chatbot, history, systemPromptTxt, WEB_PORT):
|
||||||
history = [] # 清空历史,以免输入溢出
|
history = [] # 清空历史,以免输入溢出
|
||||||
import glob, os
|
import glob, os
|
||||||
if os.path.exists(txt):
|
if os.path.exists(txt):
|
||||||
project_folder = txt
|
project_folder = txt
|
||||||
else:
|
else:
|
||||||
if txt == "": txt = '空空如也的输入栏'
|
if txt == "": txt = '空空如也的输入栏'
|
||||||
report_execption(chatbot, history, a = f"解析项目: {txt}", b = f"找不到本地项目或无权访问: {txt}")
|
report_execption(chatbot, history, a=f"解析项目: {txt}", b=f"找不到本地项目或无权访问: {txt}")
|
||||||
|
yield chatbot, history, '正常'
|
||||||
|
return
|
||||||
|
file_manifest = [f for f in glob.glob(f'{project_folder}/**/*.java', recursive=True)] + \
|
||||||
|
[f for f in glob.glob(f'{project_folder}/**/*.jar', recursive=True)] + \
|
||||||
|
[f for f in glob.glob(f'{project_folder}/**/*.xml', recursive=True)] + \
|
||||||
|
[f for f in glob.glob(f'{project_folder}/**/*.sh', recursive=True)]
|
||||||
|
if len(file_manifest) == 0:
|
||||||
|
report_execption(chatbot, history, a=f"解析项目: {txt}", b=f"找不到任何java文件: {txt}")
|
||||||
|
yield chatbot, history, '正常'
|
||||||
|
return
|
||||||
|
yield from 解析源代码(file_manifest, project_folder, top_p, temperature, chatbot, history, systemPromptTxt)
|
||||||
|
|
||||||
|
|
||||||
|
@CatchException
|
||||||
|
def 解析一个Rect项目(txt, top_p, temperature, chatbot, history, systemPromptTxt, WEB_PORT):
|
||||||
|
history = [] # 清空历史,以免输入溢出
|
||||||
|
import glob, os
|
||||||
|
if os.path.exists(txt):
|
||||||
|
project_folder = txt
|
||||||
|
else:
|
||||||
|
if txt == "": txt = '空空如也的输入栏'
|
||||||
|
report_execption(chatbot, history, a=f"解析项目: {txt}", b=f"找不到本地项目或无权访问: {txt}")
|
||||||
|
yield chatbot, history, '正常'
|
||||||
|
return
|
||||||
|
file_manifest = [f for f in glob.glob(f'{project_folder}/**/*.ts', recursive=True)] + \
|
||||||
|
[f for f in glob.glob(f'{project_folder}/**/*.tsx', recursive=True)] + \
|
||||||
|
[f for f in glob.glob(f'{project_folder}/**/*.json', recursive=True)] + \
|
||||||
|
[f for f in glob.glob(f'{project_folder}/**/*.js', recursive=True)] + \
|
||||||
|
[f for f in glob.glob(f'{project_folder}/**/*.jsx', recursive=True)]
|
||||||
|
if len(file_manifest) == 0:
|
||||||
|
report_execption(chatbot, history, a=f"解析项目: {txt}", b=f"找不到任何Rect文件: {txt}")
|
||||||
|
yield chatbot, history, '正常'
|
||||||
|
return
|
||||||
|
yield from 解析源代码(file_manifest, project_folder, top_p, temperature, chatbot, history, systemPromptTxt)
|
||||||
|
|
||||||
|
|
||||||
|
@CatchException
|
||||||
|
def 解析一个Golang项目(txt, top_p, temperature, chatbot, history, systemPromptTxt, WEB_PORT):
|
||||||
|
history = [] # 清空历史,以免输入溢出
|
||||||
|
import glob, os
|
||||||
|
if os.path.exists(txt):
|
||||||
|
project_folder = txt
|
||||||
|
else:
|
||||||
|
if txt == "": txt = '空空如也的输入栏'
|
||||||
|
report_execption(chatbot, history, a=f"解析项目: {txt}", b=f"找不到本地项目或无权访问: {txt}")
|
||||||
yield chatbot, history, '正常'
|
yield chatbot, history, '正常'
|
||||||
return
|
return
|
||||||
file_manifest = [f for f in glob.glob(f'{project_folder}/**/*.go', recursive=True)]
|
file_manifest = [f for f in glob.glob(f'{project_folder}/**/*.go', recursive=True)]
|
||||||
if len(file_manifest) == 0:
|
if len(file_manifest) == 0:
|
||||||
report_execption(chatbot, history, a = f"解析项目: {txt}", b = f"找不到任何golang文件: {txt}")
|
report_execption(chatbot, history, a=f"解析项目: {txt}", b=f"找不到任何golang文件: {txt}")
|
||||||
yield chatbot, history, '正常'
|
yield chatbot, history, '正常'
|
||||||
return
|
return
|
||||||
yield from 解析源代码(file_manifest, project_folder, top_p, temperature, chatbot, history, systemPromptTxt)
|
yield from 解析源代码(file_manifest, project_folder, top_p, temperature, chatbot, history, systemPromptTxt)
|
||||||
|
|||||||
+52
-34
@@ -1,13 +1,8 @@
|
|||||||
from toolbox import HotReload # HotReload 的意思是热更新,修改函数插件后,不需要重启程序,代码直接生效
|
from toolbox import HotReload # HotReload 的意思是热更新,修改函数插件后,不需要重启程序,代码直接生效
|
||||||
|
|
||||||
# UserVisibleLevel是过滤器参数。
|
|
||||||
# 由于UI界面空间有限,所以通过这种方式决定UI界面中显示哪些插件
|
|
||||||
# 默认函数插件 VisibleLevel 是 0
|
|
||||||
# 当 UserVisibleLevel >= 函数插件的 VisibleLevel 时,该函数插件才会被显示出来
|
|
||||||
UserVisibleLevel = 1
|
|
||||||
|
|
||||||
|
|
||||||
def get_crazy_functionals():
|
def get_crazy_functionals():
|
||||||
|
###################### 第一组插件 ###########################
|
||||||
|
# [第一组插件]: 最早期编写的项目插件和一些demo
|
||||||
from crazy_functions.读文章写摘要 import 读文章写摘要
|
from crazy_functions.读文章写摘要 import 读文章写摘要
|
||||||
from crazy_functions.生成函数注释 import 批量生成函数注释
|
from crazy_functions.生成函数注释 import 批量生成函数注释
|
||||||
from crazy_functions.解析项目源代码 import 解析项目本身
|
from crazy_functions.解析项目源代码 import 解析项目本身
|
||||||
@@ -15,39 +10,51 @@ def get_crazy_functionals():
|
|||||||
from crazy_functions.解析项目源代码 import 解析一个C项目的头文件
|
from crazy_functions.解析项目源代码 import 解析一个C项目的头文件
|
||||||
from crazy_functions.解析项目源代码 import 解析一个C项目
|
from crazy_functions.解析项目源代码 import 解析一个C项目
|
||||||
from crazy_functions.解析项目源代码 import 解析一个Golang项目
|
from crazy_functions.解析项目源代码 import 解析一个Golang项目
|
||||||
|
from crazy_functions.解析项目源代码 import 解析一个Java项目
|
||||||
|
from crazy_functions.解析项目源代码 import 解析一个Rect项目
|
||||||
from crazy_functions.高级功能函数模板 import 高阶功能模板函数
|
from crazy_functions.高级功能函数模板 import 高阶功能模板函数
|
||||||
from crazy_functions.代码重写为全英文_多线程 import 全项目切换英文
|
from crazy_functions.代码重写为全英文_多线程 import 全项目切换英文
|
||||||
|
|
||||||
function_plugins = {
|
function_plugins = {
|
||||||
"请解析并解构此项目本身(源码自译解)": {
|
"请解析并解构此项目本身(源码自译解)": {
|
||||||
"AsButton": False, # 加入下拉菜单中
|
"AsButton": False, # 加入下拉菜单中
|
||||||
"Function": 解析项目本身
|
"Function": HotReload(解析项目本身)
|
||||||
},
|
},
|
||||||
"解析整个Py项目": {
|
"解析整个Py项目": {
|
||||||
"Color": "stop", # 按钮颜色
|
"Color": "stop", # 按钮颜色
|
||||||
"Function": 解析一个Python项目
|
"Function": HotReload(解析一个Python项目)
|
||||||
},
|
},
|
||||||
"解析整个C++项目头文件": {
|
"解析整个C++项目头文件": {
|
||||||
"Color": "stop", # 按钮颜色
|
"Color": "stop", # 按钮颜色
|
||||||
"Function": 解析一个C项目的头文件
|
"Function": HotReload(解析一个C项目的头文件)
|
||||||
},
|
},
|
||||||
"解析整个C++项目(.cpp/.h)": {
|
"解析整个C++项目(.cpp/.h)": {
|
||||||
"Color": "stop", # 按钮颜色
|
"Color": "stop", # 按钮颜色
|
||||||
"AsButton": False, # 加入下拉菜单中
|
"AsButton": False, # 加入下拉菜单中
|
||||||
"Function": 解析一个C项目
|
"Function": HotReload(解析一个C项目)
|
||||||
},
|
},
|
||||||
"解析整个Go项目": {
|
"解析整个Go项目": {
|
||||||
"Color": "stop", # 按钮颜色
|
"Color": "stop", # 按钮颜色
|
||||||
"AsButton": False, # 加入下拉菜单中
|
"AsButton": False, # 加入下拉菜单中
|
||||||
"Function": 解析一个Golang项目
|
"Function": HotReload(解析一个Golang项目)
|
||||||
|
},
|
||||||
|
"解析整个Java项目": {
|
||||||
|
"Color": "stop", # 按钮颜色
|
||||||
|
"AsButton": False, # 加入下拉菜单中
|
||||||
|
"Function": HotReload(解析一个Java项目)
|
||||||
|
},
|
||||||
|
"解析整个React项目": {
|
||||||
|
"Color": "stop", # 按钮颜色
|
||||||
|
"AsButton": False, # 加入下拉菜单中
|
||||||
|
"Function": HotReload(解析一个Rect项目)
|
||||||
},
|
},
|
||||||
"读Tex论文写摘要": {
|
"读Tex论文写摘要": {
|
||||||
"Color": "stop", # 按钮颜色
|
"Color": "stop", # 按钮颜色
|
||||||
"Function": 读文章写摘要
|
"Function": HotReload(读文章写摘要)
|
||||||
},
|
},
|
||||||
"批量生成函数注释": {
|
"批量生成函数注释": {
|
||||||
"Color": "stop", # 按钮颜色
|
"Color": "stop", # 按钮颜色
|
||||||
"Function": 批量生成函数注释
|
"Function": HotReload(批量生成函数注释)
|
||||||
},
|
},
|
||||||
"[多线程demo] 把本项目源代码切换成全英文": {
|
"[多线程demo] 把本项目源代码切换成全英文": {
|
||||||
# HotReload 的意思是热更新,修改函数插件代码后,不需要重启程序,代码直接生效
|
# HotReload 的意思是热更新,修改函数插件代码后,不需要重启程序,代码直接生效
|
||||||
@@ -58,33 +65,44 @@ def get_crazy_functionals():
|
|||||||
"Function": HotReload(高阶功能模板函数)
|
"Function": HotReload(高阶功能模板函数)
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
###################### 第二组插件 ###########################
|
||||||
|
# [第二组插件]: 经过充分测试,但功能上距离达到完美状态还差一点点
|
||||||
|
from crazy_functions.批量总结PDF文档 import 批量总结PDF文档
|
||||||
|
from crazy_functions.批量总结PDF文档pdfminer import 批量总结PDF文档pdfminer
|
||||||
|
from crazy_functions.总结word文档 import 总结word文档
|
||||||
|
function_plugins.update({
|
||||||
|
"[仅供开发调试] 批量总结PDF文档": {
|
||||||
|
"Color": "stop",
|
||||||
|
"Function": HotReload(批量总结PDF文档) # HotReload 的意思是热更新,修改函数插件代码后,不需要重启程序,代码直接生效
|
||||||
|
},
|
||||||
|
"[仅供开发调试] 批量总结PDF文档pdfminer": {
|
||||||
|
"Color": "stop",
|
||||||
|
"AsButton": False, # 加入下拉菜单中
|
||||||
|
"Function": HotReload(批量总结PDF文档pdfminer)
|
||||||
|
},
|
||||||
|
"[仅供开发调试] 批量总结Word文档": {
|
||||||
|
"Color": "stop",
|
||||||
|
"Function": HotReload(总结word文档)
|
||||||
|
},
|
||||||
|
})
|
||||||
|
|
||||||
# VisibleLevel=1 经过测试,但功能上距离达到完美状态还差一点点
|
###################### 第三组插件 ###########################
|
||||||
if UserVisibleLevel >= 1:
|
# [第三组插件]: 尚未充分测试的函数插件,放在这里
|
||||||
from crazy_functions.批量总结PDF文档 import 批量总结PDF文档
|
try:
|
||||||
from crazy_functions.批量总结PDF文档pdfminer import 批量总结PDF文档pdfminer
|
from crazy_functions.下载arxiv论文翻译摘要 import 下载arxiv论文并翻译摘要
|
||||||
from crazy_functions.总结word文档 import 总结word文档
|
|
||||||
function_plugins.update({
|
function_plugins.update({
|
||||||
"[仅供开发调试] 批量总结PDF文档": {
|
"一键下载arxiv论文并翻译摘要(先在input输入编号,如1812.10695)": {
|
||||||
"Color": "stop",
|
|
||||||
"Function": HotReload(批量总结PDF文档) # HotReload 的意思是热更新,修改函数插件代码后,不需要重启程序,代码直接生效
|
|
||||||
},
|
|
||||||
"[仅供开发调试] 批量总结PDF文档pdfminer": {
|
|
||||||
"Color": "stop",
|
"Color": "stop",
|
||||||
"AsButton": False, # 加入下拉菜单中
|
"AsButton": False, # 加入下拉菜单中
|
||||||
"Function": HotReload(批量总结PDF文档pdfminer)
|
"Function": HotReload(下载arxiv论文并翻译摘要)
|
||||||
},
|
}
|
||||||
"[仅供开发调试] 批量总结Word文档": {
|
|
||||||
"Color": "stop",
|
|
||||||
"Function": HotReload(总结word文档)
|
|
||||||
},
|
|
||||||
})
|
})
|
||||||
|
except Exception as err:
|
||||||
|
print(f'[下载arxiv论文并翻译摘要] 插件导入失败 {str(err)}')
|
||||||
|
|
||||||
# VisibleLevel=2 尚未充分测试的函数插件,放在这里
|
|
||||||
if UserVisibleLevel >= 2:
|
|
||||||
function_plugins.update({
|
|
||||||
})
|
|
||||||
|
|
||||||
|
|
||||||
|
###################### 第n组插件 ###########################
|
||||||
return function_plugins
|
return function_plugins
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,19 +1,19 @@
|
|||||||
import os; os.environ['no_proxy'] = '*' # 避免代理网络产生意外污染
|
import os; os.environ['no_proxy'] = '*' # 避免代理网络产生意外污染
|
||||||
import gradio as gr
|
import gradio as gr
|
||||||
from predict import predict
|
from predict import predict
|
||||||
from toolbox import format_io, find_free_port, on_file_uploaded, on_report_generated, get_conf
|
from toolbox import format_io, find_free_port, on_file_uploaded, on_report_generated, get_conf, ArgsGeneralWrapper, DummyWith
|
||||||
|
|
||||||
# 建议您复制一个config_private.py放自己的秘密, 如API和代理网址, 避免不小心传github被别人看到
|
# 建议您复制一个config_private.py放自己的秘密, 如API和代理网址, 避免不小心传github被别人看到
|
||||||
proxies, WEB_PORT, LLM_MODEL, CONCURRENT_COUNT, AUTHENTICATION, CHATBOT_HEIGHT = \
|
proxies, WEB_PORT, LLM_MODEL, CONCURRENT_COUNT, AUTHENTICATION, CHATBOT_HEIGHT, LAYOUT = \
|
||||||
get_conf('proxies', 'WEB_PORT', 'LLM_MODEL', 'CONCURRENT_COUNT', 'AUTHENTICATION', 'CHATBOT_HEIGHT')
|
get_conf('proxies', 'WEB_PORT', 'LLM_MODEL', 'CONCURRENT_COUNT', 'AUTHENTICATION', 'CHATBOT_HEIGHT', 'LAYOUT')
|
||||||
|
|
||||||
# 如果WEB_PORT是-1, 则随机选取WEB端口
|
# 如果WEB_PORT是-1, 则随机选取WEB端口
|
||||||
PORT = find_free_port() if WEB_PORT <= 0 else WEB_PORT
|
PORT = find_free_port() if WEB_PORT <= 0 else WEB_PORT
|
||||||
if not AUTHENTICATION: AUTHENTICATION = None
|
if not AUTHENTICATION: AUTHENTICATION = None
|
||||||
|
|
||||||
title = "ChatGPT 学术优化" if LLM_MODEL.startswith('gpt') else "ChatGPT / LLM 学术优化"
|
|
||||||
initial_prompt = "Serve me as a writing and programming assistant."
|
initial_prompt = "Serve me as a writing and programming assistant."
|
||||||
title_html = f"<h1 align=\"center\">{title}</h1>"
|
title_html = "<h1 align=\"center\">ChatGPT 学术优化</h1>"
|
||||||
|
description = """代码开源和更新[地址🚀](https://github.com/binary-husky/chatgpt_academic),感谢热情的[开发者们❤️](https://github.com/binary-husky/chatgpt_academic/graphs/contributors)"""
|
||||||
|
|
||||||
# 问询记录, python 版本建议3.9+(越新越好)
|
# 问询记录, python 版本建议3.9+(越新越好)
|
||||||
import logging
|
import logging
|
||||||
@@ -37,25 +37,36 @@ gr.Chatbot.postprocess = format_io
|
|||||||
from theme import adjust_theme, advanced_css
|
from theme import adjust_theme, advanced_css
|
||||||
set_theme = adjust_theme()
|
set_theme = adjust_theme()
|
||||||
|
|
||||||
|
# 代理与自动更新
|
||||||
|
from check_proxy import check_proxy, auto_update
|
||||||
|
proxy_info = check_proxy(proxies)
|
||||||
|
|
||||||
|
gr_L1 = lambda: gr.Row().style()
|
||||||
|
gr_L2 = lambda scale: gr.Column(scale=scale)
|
||||||
|
if LAYOUT == "TOP-DOWN":
|
||||||
|
gr_L1 = lambda: DummyWith()
|
||||||
|
gr_L2 = lambda scale: gr.Row()
|
||||||
|
CHATBOT_HEIGHT /= 2
|
||||||
|
|
||||||
cancel_handles = []
|
cancel_handles = []
|
||||||
with gr.Blocks(theme=set_theme, analytics_enabled=False, css=advanced_css) as demo:
|
with gr.Blocks(theme=set_theme, analytics_enabled=False, css=advanced_css) as demo:
|
||||||
gr.HTML(title_html)
|
gr.HTML(title_html)
|
||||||
with gr.Row().style(equal_height=True):
|
with gr_L1():
|
||||||
with gr.Column(scale=2):
|
with gr_L2(scale=2):
|
||||||
chatbot = gr.Chatbot()
|
chatbot = gr.Chatbot()
|
||||||
chatbot.style(height=CHATBOT_HEIGHT)
|
chatbot.style(height=CHATBOT_HEIGHT)
|
||||||
history = gr.State([])
|
history = gr.State([])
|
||||||
with gr.Column(scale=1):
|
with gr_L2(scale=1):
|
||||||
with gr.Row():
|
with gr.Accordion("输入区", open=True) as area_input_primary:
|
||||||
txt = gr.Textbox(show_label=False, placeholder="Input question here.").style(container=False)
|
with gr.Row():
|
||||||
with gr.Row():
|
txt = gr.Textbox(show_label=False, placeholder="Input question here.").style(container=False)
|
||||||
submitBtn = gr.Button("提交", variant="primary")
|
with gr.Row():
|
||||||
with gr.Row():
|
submitBtn = gr.Button("提交", variant="primary")
|
||||||
resetBtn = gr.Button("重置", variant="secondary"); resetBtn.style(size="sm")
|
with gr.Row():
|
||||||
stopBtn = gr.Button("停止", variant="secondary"); stopBtn.style(size="sm")
|
resetBtn = gr.Button("重置", variant="secondary"); resetBtn.style(size="sm")
|
||||||
with gr.Row():
|
stopBtn = gr.Button("停止", variant="secondary"); stopBtn.style(size="sm")
|
||||||
from check_proxy import check_proxy
|
with gr.Row():
|
||||||
status = gr.Markdown(f"Tip: 按Enter提交, 按Shift+Enter换行。当前模型: {LLM_MODEL} \n {check_proxy(proxies)}")
|
status = gr.Markdown(f"Tip: 按Enter提交, 按Shift+Enter换行。当前模型: {LLM_MODEL} \n {proxy_info}")
|
||||||
with gr.Accordion("基础功能区", open=True) as area_basic_fn:
|
with gr.Accordion("基础功能区", open=True) as area_basic_fn:
|
||||||
with gr.Row():
|
with gr.Row():
|
||||||
for k in functional:
|
for k in functional:
|
||||||
@@ -63,12 +74,13 @@ with gr.Blocks(theme=set_theme, analytics_enabled=False, css=advanced_css) as de
|
|||||||
functional[k]["Button"] = gr.Button(k, variant=variant)
|
functional[k]["Button"] = gr.Button(k, variant=variant)
|
||||||
with gr.Accordion("函数插件区", open=True) as area_crazy_fn:
|
with gr.Accordion("函数插件区", open=True) as area_crazy_fn:
|
||||||
with gr.Row():
|
with gr.Row():
|
||||||
gr.Markdown("注意:以下“红颜色”标识的函数插件需从input区读取路径作为参数.")
|
gr.Markdown("注意:以下“红颜色”标识的函数插件需从输入区读取路径作为参数.")
|
||||||
with gr.Row():
|
with gr.Row():
|
||||||
for k in crazy_fns:
|
for k in crazy_fns:
|
||||||
if not crazy_fns[k].get("AsButton", True): continue
|
if not crazy_fns[k].get("AsButton", True): continue
|
||||||
variant = crazy_fns[k]["Color"] if "Color" in crazy_fns[k] else "secondary"
|
variant = crazy_fns[k]["Color"] if "Color" in crazy_fns[k] else "secondary"
|
||||||
crazy_fns[k]["Button"] = gr.Button(k, variant=variant)
|
crazy_fns[k]["Button"] = gr.Button(k, variant=variant)
|
||||||
|
crazy_fns[k]["Button"].style(size="sm")
|
||||||
with gr.Row():
|
with gr.Row():
|
||||||
with gr.Accordion("更多函数插件", open=True):
|
with gr.Accordion("更多函数插件", open=True):
|
||||||
dropdown_fn_list = [k for k in crazy_fns.keys() if not crazy_fns[k].get("AsButton", True)]
|
dropdown_fn_list = [k for k in crazy_fns.keys() if not crazy_fns[k].get("AsButton", True)]
|
||||||
@@ -79,38 +91,51 @@ with gr.Blocks(theme=set_theme, analytics_enabled=False, css=advanced_css) as de
|
|||||||
with gr.Row():
|
with gr.Row():
|
||||||
with gr.Accordion("点击展开“文件上传区”。上传本地文件可供红色函数插件调用。", open=False) as area_file_up:
|
with gr.Accordion("点击展开“文件上传区”。上传本地文件可供红色函数插件调用。", open=False) as area_file_up:
|
||||||
file_upload = gr.Files(label="任何文件, 但推荐上传压缩文件(zip, tar)", file_count="multiple")
|
file_upload = gr.Files(label="任何文件, 但推荐上传压缩文件(zip, tar)", file_count="multiple")
|
||||||
with gr.Accordion("展开SysPrompt & GPT参数 & 交互界面布局", open=False):
|
with gr.Accordion("展开SysPrompt & 交互界面布局 & Github地址", open=(LAYOUT == "TOP-DOWN")):
|
||||||
system_prompt = gr.Textbox(show_label=True, placeholder=f"System Prompt", label="System prompt", value=initial_prompt)
|
system_prompt = gr.Textbox(show_label=True, placeholder=f"System Prompt", label="System prompt", value=initial_prompt)
|
||||||
top_p = gr.Slider(minimum=-0, maximum=1.0, value=1.0, step=0.01,interactive=True, label="Top-p (nucleus sampling)",)
|
top_p = gr.Slider(minimum=-0, maximum=1.0, value=1.0, step=0.01,interactive=True, label="Top-p (nucleus sampling)",)
|
||||||
temperature = gr.Slider(minimum=-0, maximum=2.0, value=1.0, step=0.01, interactive=True, label="Temperature",)
|
temperature = gr.Slider(minimum=-0, maximum=2.0, value=1.0, step=0.01, interactive=True, label="Temperature",)
|
||||||
checkboxes = gr.CheckboxGroup(["基础功能区", "函数插件区"], value=["基础功能区", "函数插件区"], label="显示/隐藏功能区")
|
checkboxes = gr.CheckboxGroup(["基础功能区", "函数插件区", "底部输入区"], value=["基础功能区", "函数插件区"], label="显示/隐藏功能区")
|
||||||
|
gr.Markdown(description)
|
||||||
|
with gr.Accordion("备选输入区", open=True, visible=False) as area_input_secondary:
|
||||||
|
with gr.Row():
|
||||||
|
txt2 = gr.Textbox(show_label=False, placeholder="Input question here.", label="输入区2").style(container=False)
|
||||||
|
with gr.Row():
|
||||||
|
submitBtn2 = gr.Button("提交", variant="primary")
|
||||||
|
with gr.Row():
|
||||||
|
resetBtn2 = gr.Button("重置", variant="secondary"); resetBtn.style(size="sm")
|
||||||
|
stopBtn2 = gr.Button("停止", variant="secondary"); stopBtn.style(size="sm")
|
||||||
# 功能区显示开关与功能区的互动
|
# 功能区显示开关与功能区的互动
|
||||||
def fn_area_visibility(a):
|
def fn_area_visibility(a):
|
||||||
ret = {}
|
ret = {}
|
||||||
ret.update({area_basic_fn: gr.update(visible=("基础功能区" in a))})
|
ret.update({area_basic_fn: gr.update(visible=("基础功能区" in a))})
|
||||||
ret.update({area_crazy_fn: gr.update(visible=("函数插件区" in a))})
|
ret.update({area_crazy_fn: gr.update(visible=("函数插件区" in a))})
|
||||||
|
ret.update({area_input_primary: gr.update(visible=("底部输入区" not in a))})
|
||||||
|
ret.update({area_input_secondary: gr.update(visible=("底部输入区" in a))})
|
||||||
|
if "底部输入区" in a: ret.update({txt: gr.update(value="")})
|
||||||
return ret
|
return ret
|
||||||
checkboxes.select(fn_area_visibility, [checkboxes], [area_basic_fn, area_crazy_fn] )
|
checkboxes.select(fn_area_visibility, [checkboxes], [area_basic_fn, area_crazy_fn, area_input_primary, area_input_secondary, txt, txt2] )
|
||||||
# 整理反复出现的控件句柄组合
|
# 整理反复出现的控件句柄组合
|
||||||
input_combo = [txt, top_p, temperature, chatbot, history, system_prompt]
|
input_combo = [txt, txt2, top_p, temperature, chatbot, history, system_prompt]
|
||||||
output_combo = [chatbot, history, status]
|
output_combo = [chatbot, history, status]
|
||||||
predict_args = dict(fn=predict, inputs=input_combo, outputs=output_combo)
|
predict_args = dict(fn=ArgsGeneralWrapper(predict), inputs=input_combo, outputs=output_combo)
|
||||||
empty_txt_args = dict(fn=lambda: "", inputs=[], outputs=[txt]) # 用于在提交后清空输入栏
|
|
||||||
# 提交按钮、重置按钮
|
# 提交按钮、重置按钮
|
||||||
cancel_handles.append(txt.submit(**predict_args)) #; txt.submit(**empty_txt_args) 在提交后清空输入栏
|
cancel_handles.append(txt.submit(**predict_args))
|
||||||
cancel_handles.append(submitBtn.click(**predict_args)) #; submitBtn.click(**empty_txt_args) 在提交后清空输入栏
|
cancel_handles.append(txt2.submit(**predict_args))
|
||||||
|
cancel_handles.append(submitBtn.click(**predict_args))
|
||||||
|
cancel_handles.append(submitBtn2.click(**predict_args))
|
||||||
resetBtn.click(lambda: ([], [], "已重置"), None, output_combo)
|
resetBtn.click(lambda: ([], [], "已重置"), None, output_combo)
|
||||||
|
resetBtn2.click(lambda: ([], [], "已重置"), None, output_combo)
|
||||||
# 基础功能区的回调函数注册
|
# 基础功能区的回调函数注册
|
||||||
for k in functional:
|
for k in functional:
|
||||||
click_handle = functional[k]["Button"].click(predict, [*input_combo, gr.State(True), gr.State(k)], output_combo)
|
click_handle = functional[k]["Button"].click(fn=ArgsGeneralWrapper(predict), inputs=[*input_combo, gr.State(True), gr.State(k)], outputs=output_combo)
|
||||||
cancel_handles.append(click_handle)
|
cancel_handles.append(click_handle)
|
||||||
# 文件上传区,接收文件后与chatbot的互动
|
# 文件上传区,接收文件后与chatbot的互动
|
||||||
file_upload.upload(on_file_uploaded, [file_upload, chatbot, txt], [chatbot, txt])
|
file_upload.upload(on_file_uploaded, [file_upload, chatbot, txt], [chatbot, txt])
|
||||||
# 函数插件-固定按钮区
|
# 函数插件-固定按钮区
|
||||||
for k in crazy_fns:
|
for k in crazy_fns:
|
||||||
if not crazy_fns[k].get("AsButton", True): continue
|
if not crazy_fns[k].get("AsButton", True): continue
|
||||||
click_handle = crazy_fns[k]["Button"].click(crazy_fns[k]["Function"], [*input_combo, gr.State(PORT)], output_combo)
|
click_handle = crazy_fns[k]["Button"].click(ArgsGeneralWrapper(crazy_fns[k]["Function"]), [*input_combo, gr.State(PORT)], output_combo)
|
||||||
click_handle.then(on_report_generated, [file_upload, chatbot], [file_upload, chatbot])
|
click_handle.then(on_report_generated, [file_upload, chatbot], [file_upload, chatbot])
|
||||||
cancel_handles.append(click_handle)
|
cancel_handles.append(click_handle)
|
||||||
# 函数插件-下拉菜单与随变按钮的互动
|
# 函数插件-下拉菜单与随变按钮的互动
|
||||||
@@ -120,8 +145,8 @@ with gr.Blocks(theme=set_theme, analytics_enabled=False, css=advanced_css) as de
|
|||||||
dropdown.select(on_dropdown_changed, [dropdown], [switchy_bt] )
|
dropdown.select(on_dropdown_changed, [dropdown], [switchy_bt] )
|
||||||
# 随变按钮的回调函数注册
|
# 随变按钮的回调函数注册
|
||||||
def route(k, *args, **kwargs):
|
def route(k, *args, **kwargs):
|
||||||
if k in [r"打开插件列表", r"先从插件列表中选择"]: return
|
if k in [r"打开插件列表", r"请先从插件列表中选择"]: return
|
||||||
yield from crazy_fns[k]["Function"](*args, **kwargs)
|
yield from ArgsGeneralWrapper(crazy_fns[k]["Function"])(*args, **kwargs)
|
||||||
click_handle = switchy_bt.click(route,[switchy_bt, *input_combo, gr.State(PORT)], output_combo)
|
click_handle = switchy_bt.click(route,[switchy_bt, *input_combo, gr.State(PORT)], output_combo)
|
||||||
click_handle.then(on_report_generated, [file_upload, chatbot], [file_upload, chatbot])
|
click_handle.then(on_report_generated, [file_upload, chatbot], [file_upload, chatbot])
|
||||||
# def expand_file_area(file_upload, area_file_up):
|
# def expand_file_area(file_upload, area_file_up):
|
||||||
@@ -130,16 +155,20 @@ with gr.Blocks(theme=set_theme, analytics_enabled=False, css=advanced_css) as de
|
|||||||
cancel_handles.append(click_handle)
|
cancel_handles.append(click_handle)
|
||||||
# 终止按钮的回调函数注册
|
# 终止按钮的回调函数注册
|
||||||
stopBtn.click(fn=None, inputs=None, outputs=None, cancels=cancel_handles)
|
stopBtn.click(fn=None, inputs=None, outputs=None, cancels=cancel_handles)
|
||||||
|
stopBtn2.click(fn=None, inputs=None, outputs=None, cancels=cancel_handles)
|
||||||
# gradio的inbrowser触发不太稳定,回滚代码到原始的浏览器打开函数
|
# gradio的inbrowser触发不太稳定,回滚代码到原始的浏览器打开函数
|
||||||
def auto_opentab_delay():
|
def auto_opentab_delay():
|
||||||
import threading, webbrowser, time
|
import threading, webbrowser, time
|
||||||
print(f"如果浏览器没有自动打开,请复制并转到以下URL: http://localhost:{PORT}")
|
print(f"如果浏览器没有自动打开,请复制并转到以下URL:")
|
||||||
|
print(f"\t(亮色主体): http://localhost:{PORT}")
|
||||||
|
print(f"\t(暗色主体): http://localhost:{PORT}/?__dark-theme=true")
|
||||||
def open():
|
def open():
|
||||||
time.sleep(2)
|
time.sleep(2)
|
||||||
webbrowser.open_new_tab(f"http://localhost:{PORT}")
|
try: auto_update() # 检查新版本
|
||||||
|
except: pass
|
||||||
|
webbrowser.open_new_tab(f"http://localhost:{PORT}/?__dark-theme=true")
|
||||||
threading.Thread(target=open, name="open-browser", daemon=True).start()
|
threading.Thread(target=open, name="open-browser", daemon=True).start()
|
||||||
|
|
||||||
auto_opentab_delay()
|
auto_opentab_delay()
|
||||||
demo.title = title
|
demo.title = "ChatGPT 学术优化"
|
||||||
demo.queue(concurrency_count=CONCURRENT_COUNT).launch(server_name="0.0.0.0", share=True, server_port=PORT, auth=AUTHENTICATION)
|
demo.queue(concurrency_count=CONCURRENT_COUNT).launch(server_name="0.0.0.0", share=True, server_port=PORT, auth=AUTHENTICATION)
|
||||||
|
|||||||
+14
-15
@@ -71,9 +71,10 @@ def predict_no_ui(inputs, top_p, temperature, history=[], sys_prompt=""):
|
|||||||
raise ConnectionAbortedError("Json解析不合常规,可能是文本过长" + response.text)
|
raise ConnectionAbortedError("Json解析不合常规,可能是文本过长" + response.text)
|
||||||
|
|
||||||
|
|
||||||
def predict_no_ui_long_connection(inputs, top_p, temperature, history=[], sys_prompt=""):
|
def predict_no_ui_long_connection(inputs, top_p, temperature, history=[], sys_prompt="", observe_window=None):
|
||||||
"""
|
"""
|
||||||
发送至chatGPT,等待回复,一次性完成,不显示中间过程。但内部用stream的方法避免有人中途掐网线。
|
发送至chatGPT,等待回复,一次性完成,不显示中间过程。但内部用stream的方法避免有人中途掐网线。
|
||||||
|
observe_window:用于负责跨越线程传递已经输出的部分,大部分时候仅仅为了fancy的视觉效果,留空即可
|
||||||
"""
|
"""
|
||||||
headers, payload = generate_payload(inputs, top_p, temperature, history, system_prompt=sys_prompt, stream=True)
|
headers, payload = generate_payload(inputs, top_p, temperature, history, system_prompt=sys_prompt, stream=True)
|
||||||
|
|
||||||
@@ -105,14 +106,18 @@ def predict_no_ui_long_connection(inputs, top_p, temperature, history=[], sys_pr
|
|||||||
delta = json_data["delta"]
|
delta = json_data["delta"]
|
||||||
if len(delta) == 0: break
|
if len(delta) == 0: break
|
||||||
if "role" in delta: continue
|
if "role" in delta: continue
|
||||||
if "content" in delta: result += delta["content"]; print(delta["content"], end='')
|
if "content" in delta:
|
||||||
|
result += delta["content"]
|
||||||
|
print(delta["content"], end='')
|
||||||
|
if observe_window is not None: observe_window[0] += delta["content"]
|
||||||
else: raise RuntimeError("意外Json结构:"+delta)
|
else: raise RuntimeError("意外Json结构:"+delta)
|
||||||
if json_data['finish_reason'] == 'length':
|
if json_data['finish_reason'] == 'length':
|
||||||
raise ConnectionAbortedError("正常结束,但显示Token不足。")
|
raise ConnectionAbortedError("正常结束,但显示Token不足。")
|
||||||
return result
|
return result
|
||||||
|
|
||||||
|
|
||||||
def predict(inputs, top_p, temperature, chatbot=[], history=[], system_prompt='', stream = True, additional_fn=None):
|
def predict(inputs, top_p, temperature, chatbot=[], history=[], system_prompt='',
|
||||||
|
stream = True, additional_fn=None):
|
||||||
"""
|
"""
|
||||||
发送至chatGPT,流式获取输出。
|
发送至chatGPT,流式获取输出。
|
||||||
用于基础的对话功能。
|
用于基础的对话功能。
|
||||||
@@ -185,14 +190,16 @@ def predict(inputs, top_p, temperature, chatbot=[], history=[], system_prompt=''
|
|||||||
error_msg = chunk.decode()
|
error_msg = chunk.decode()
|
||||||
if "reduce the length" in error_msg:
|
if "reduce the length" in error_msg:
|
||||||
chatbot[-1] = (chatbot[-1][0], "[Local Message] Input (or history) is too long, please reduce input or clear history by refreshing this page.")
|
chatbot[-1] = (chatbot[-1][0], "[Local Message] Input (or history) is too long, please reduce input or clear history by refreshing this page.")
|
||||||
history = []
|
history = [] # 清除历史
|
||||||
elif "Incorrect API key" in error_msg:
|
elif "Incorrect API key" in error_msg:
|
||||||
chatbot[-1] = (chatbot[-1][0], "[Local Message] Incorrect API key provided.")
|
chatbot[-1] = (chatbot[-1][0], "[Local Message] Incorrect API key provided.")
|
||||||
|
elif "exceeded your current quota" in error_msg:
|
||||||
|
chatbot[-1] = (chatbot[-1][0], "[Local Message] You exceeded your current quota. OpenAI以账户额度不足为由,拒绝服务.")
|
||||||
else:
|
else:
|
||||||
from toolbox import regular_txt_to_markdown
|
from toolbox import regular_txt_to_markdown
|
||||||
tb_str = regular_txt_to_markdown(traceback.format_exc())
|
tb_str = '```\n' + traceback.format_exc() + '```'
|
||||||
chatbot[-1] = (chatbot[-1][0], f"[Local Message] Json Error \n\n {tb_str} \n\n {regular_txt_to_markdown(chunk.decode()[4:])}")
|
chatbot[-1] = (chatbot[-1][0], f"[Local Message] 异常 \n\n{tb_str} \n\n{regular_txt_to_markdown(chunk.decode()[4:])}")
|
||||||
yield chatbot, history, "Json解析不合常规" + error_msg
|
yield chatbot, history, "Json异常" + error_msg
|
||||||
return
|
return
|
||||||
|
|
||||||
def generate_payload(inputs, top_p, temperature, history, system_prompt, stream):
|
def generate_payload(inputs, top_p, temperature, history, system_prompt, stream):
|
||||||
@@ -243,11 +250,3 @@ def generate_payload(inputs, top_p, temperature, history, system_prompt, stream)
|
|||||||
return headers,payload
|
return headers,payload
|
||||||
|
|
||||||
|
|
||||||
if not LLM_MODEL.startswith('gpt'):
|
|
||||||
# 函数重载到另一个文件
|
|
||||||
from request_llm.bridge_tgui import predict_tgui, predict_tgui_no_ui
|
|
||||||
predict = predict_tgui
|
|
||||||
predict_no_ui = predict_tgui_no_ui
|
|
||||||
predict_no_ui_long_connection = predict_tgui_no_ui
|
|
||||||
|
|
||||||
|
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
# 如何使用其他大语言模型
|
# 如何使用其他大语言模型(dev分支测试中)
|
||||||
|
|
||||||
## 1. 先运行text-generation
|
## 1. 先运行text-generation
|
||||||
``` sh
|
``` sh
|
||||||
|
|||||||
@@ -3,3 +3,5 @@ requests[socks]
|
|||||||
mdtex2html
|
mdtex2html
|
||||||
Markdown
|
Markdown
|
||||||
latex2mathml
|
latex2mathml
|
||||||
|
openai
|
||||||
|
transformers
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ import gradio as gr
|
|||||||
|
|
||||||
def adjust_theme():
|
def adjust_theme():
|
||||||
try:
|
try:
|
||||||
color_er = gr.themes.utils.colors.pink
|
color_er = gr.themes.utils.colors.fuchsia
|
||||||
set_theme = gr.themes.Default(
|
set_theme = gr.themes.Default(
|
||||||
primary_hue=gr.themes.utils.colors.orange,
|
primary_hue=gr.themes.utils.colors.orange,
|
||||||
neutral_hue=gr.themes.utils.colors.gray,
|
neutral_hue=gr.themes.utils.colors.gray,
|
||||||
@@ -82,13 +82,71 @@ def adjust_theme():
|
|||||||
return set_theme
|
return set_theme
|
||||||
|
|
||||||
advanced_css = """
|
advanced_css = """
|
||||||
|
/* 设置表格的外边距为1em,内部单元格之间边框合并,空单元格显示. */
|
||||||
.markdown-body table {
|
.markdown-body table {
|
||||||
border: 1px solid #ddd;
|
margin: 1em 0;
|
||||||
border-collapse: collapse;
|
border-collapse: collapse;
|
||||||
|
empty-cells: show;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* 设置表格单元格的内边距为5px,边框粗细为1.2px,颜色为--border-color-primary. */
|
||||||
.markdown-body th, .markdown-body td {
|
.markdown-body th, .markdown-body td {
|
||||||
border: 1px solid #ddd;
|
border: 1.2px solid var(--border-color-primary);
|
||||||
padding: 5px;
|
padding: 5px;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* 设置表头背景颜色为rgba(175,184,193,0.2),透明度为0.2. */
|
||||||
|
.markdown-body thead {
|
||||||
|
background-color: rgba(175,184,193,0.2);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* 设置表头单元格的内边距为0.5em和0.2em. */
|
||||||
|
.markdown-body thead th {
|
||||||
|
padding: .5em .2em;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* 去掉列表前缀的默认间距,使其与文本线对齐. */
|
||||||
|
.markdown-body ol, .markdown-body ul {
|
||||||
|
padding-inline-start: 2em !important;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* 设定聊天气泡的样式,包括圆角、最大宽度和阴影等. */
|
||||||
|
[class *= "message"] {
|
||||||
|
border-radius: var(--radius-xl) !important;
|
||||||
|
/* padding: var(--spacing-xl) !important; */
|
||||||
|
/* font-size: var(--text-md) !important; */
|
||||||
|
/* line-height: var(--line-md) !important; */
|
||||||
|
/* min-height: calc(var(--text-md)*var(--line-md) + 2*var(--spacing-xl)); */
|
||||||
|
/* min-width: calc(var(--text-md)*var(--line-md) + 2*var(--spacing-xl)); */
|
||||||
|
}
|
||||||
|
[data-testid = "bot"] {
|
||||||
|
max-width: 95%;
|
||||||
|
/* width: auto !important; */
|
||||||
|
border-bottom-left-radius: 0 !important;
|
||||||
|
}
|
||||||
|
[data-testid = "user"] {
|
||||||
|
max-width: 100%;
|
||||||
|
/* width: auto !important; */
|
||||||
|
border-bottom-right-radius: 0 !important;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* 行内代码的背景设为淡灰色,设定圆角和间距. */
|
||||||
|
.markdown-body code {
|
||||||
|
display: inline;
|
||||||
|
white-space: break-spaces;
|
||||||
|
border-radius: 6px;
|
||||||
|
margin: 0 2px 0 2px;
|
||||||
|
padding: .2em .4em .1em .4em;
|
||||||
|
background-color: rgba(175,184,193,0.2);
|
||||||
|
}
|
||||||
|
/* 设定代码块的样式,包括背景颜色、内、外边距、圆角。 */
|
||||||
|
.markdown-body pre code {
|
||||||
|
display: block;
|
||||||
|
overflow: auto;
|
||||||
|
white-space: pre;
|
||||||
|
background-color: rgba(175,184,193,0.2);
|
||||||
|
border-radius: 10px;
|
||||||
|
padding: 1em;
|
||||||
|
margin: 1em 2em 1em 0.5em;
|
||||||
|
}
|
||||||
"""
|
"""
|
||||||
+39
-3
@@ -2,6 +2,17 @@ import markdown, mdtex2html, threading, importlib, traceback, importlib, inspect
|
|||||||
from show_math import convert as convert_math
|
from show_math import convert as convert_math
|
||||||
from functools import wraps, lru_cache
|
from functools import wraps, lru_cache
|
||||||
|
|
||||||
|
def ArgsGeneralWrapper(f):
|
||||||
|
"""
|
||||||
|
装饰器函数,用于重组输入参数,改变输入参数的顺序与结构。
|
||||||
|
"""
|
||||||
|
def decorated(txt, txt2, *args, **kwargs):
|
||||||
|
txt_passon = txt
|
||||||
|
if txt == "" and txt2 != "": txt_passon = txt2
|
||||||
|
yield from f(txt_passon, *args, **kwargs)
|
||||||
|
return decorated
|
||||||
|
|
||||||
|
|
||||||
def get_reduce_token_percent(text):
|
def get_reduce_token_percent(text):
|
||||||
try:
|
try:
|
||||||
# text = "maximum context length is 4097 tokens. However, your messages resulted in 4870 tokens"
|
# text = "maximum context length is 4097 tokens. However, your messages resulted in 4870 tokens"
|
||||||
@@ -115,8 +126,9 @@ def CatchException(f):
|
|||||||
from check_proxy import check_proxy
|
from check_proxy import check_proxy
|
||||||
from toolbox import get_conf
|
from toolbox import get_conf
|
||||||
proxies, = get_conf('proxies')
|
proxies, = get_conf('proxies')
|
||||||
tb_str = regular_txt_to_markdown(traceback.format_exc())
|
tb_str = '```\n' + traceback.format_exc() + '```'
|
||||||
chatbot[-1] = (chatbot[-1][0], f"[Local Message] 实验性函数调用出错: \n\n {tb_str} \n\n 当前代理可用性: \n\n {check_proxy(proxies)}")
|
if chatbot is None or len(chatbot) == 0: chatbot = [["插件调度异常","异常原因"]]
|
||||||
|
chatbot[-1] = (chatbot[-1][0], f"[Local Message] 实验性函数调用出错: \n\n{tb_str} \n\n当前代理可用性: \n\n{check_proxy(proxies)}")
|
||||||
yield chatbot, history, f'异常 {e}'
|
yield chatbot, history, f'异常 {e}'
|
||||||
return decorated
|
return decorated
|
||||||
|
|
||||||
@@ -164,6 +176,23 @@ def markdown_convertion(txt):
|
|||||||
else:
|
else:
|
||||||
return pre + markdown.markdown(txt,extensions=['fenced_code','tables']) + suf
|
return pre + markdown.markdown(txt,extensions=['fenced_code','tables']) + suf
|
||||||
|
|
||||||
|
def close_up_code_segment_during_stream(gpt_reply):
|
||||||
|
"""
|
||||||
|
在gpt输出代码的中途(输出了前面的```,但还没输出完后面的```),补上后面的```
|
||||||
|
"""
|
||||||
|
if '```' not in gpt_reply: return gpt_reply
|
||||||
|
if gpt_reply.endswith('```'): return gpt_reply
|
||||||
|
|
||||||
|
# 排除了以上两个情况,我们
|
||||||
|
segments = gpt_reply.split('```')
|
||||||
|
n_mark = len(segments) - 1
|
||||||
|
if n_mark % 2 == 1:
|
||||||
|
# print('输出代码片段中!')
|
||||||
|
return gpt_reply+'\n```'
|
||||||
|
else:
|
||||||
|
return gpt_reply
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
def format_io(self, y):
|
def format_io(self, y):
|
||||||
"""
|
"""
|
||||||
@@ -172,6 +201,7 @@ def format_io(self, y):
|
|||||||
if y is None or y == []: return []
|
if y is None or y == []: return []
|
||||||
i_ask, gpt_reply = y[-1]
|
i_ask, gpt_reply = y[-1]
|
||||||
i_ask = text_divide_paragraph(i_ask) # 输入部分太自由,预处理一波
|
i_ask = text_divide_paragraph(i_ask) # 输入部分太自由,预处理一波
|
||||||
|
gpt_reply = close_up_code_segment_during_stream(gpt_reply) # 当代码输出半截的时候,试着补上后个```
|
||||||
y[-1] = (
|
y[-1] = (
|
||||||
None if i_ask is None else markdown.markdown(i_ask, extensions=['fenced_code','tables']),
|
None if i_ask is None else markdown.markdown(i_ask, extensions=['fenced_code','tables']),
|
||||||
None if gpt_reply is None else markdown_convertion(gpt_reply)
|
None if gpt_reply is None else markdown_convertion(gpt_reply)
|
||||||
@@ -284,7 +314,7 @@ def on_file_uploaded(files, chatbot, txt):
|
|||||||
def on_report_generated(files, chatbot):
|
def on_report_generated(files, chatbot):
|
||||||
from toolbox import find_recent_files
|
from toolbox import find_recent_files
|
||||||
report_files = find_recent_files('gpt_log')
|
report_files = find_recent_files('gpt_log')
|
||||||
if len(report_files) == 0: return report_files, chatbot
|
if len(report_files) == 0: return files, chatbot
|
||||||
# files.extend(report_files)
|
# files.extend(report_files)
|
||||||
chatbot.append(['汇总报告如何远程获取?', '汇总报告已经添加到右侧“文件上传区”(可能处于折叠状态),请查收。'])
|
chatbot.append(['汇总报告如何远程获取?', '汇总报告已经添加到右侧“文件上传区”(可能处于折叠状态),请查收。'])
|
||||||
return report_files, chatbot
|
return report_files, chatbot
|
||||||
@@ -323,3 +353,9 @@ def clear_line_break(txt):
|
|||||||
txt = txt.replace(' ', ' ')
|
txt = txt.replace(' ', ' ')
|
||||||
txt = txt.replace(' ', ' ')
|
txt = txt.replace(' ', ' ')
|
||||||
return txt
|
return txt
|
||||||
|
|
||||||
|
class DummyWith():
|
||||||
|
def __enter__(self):
|
||||||
|
return self
|
||||||
|
def __exit__(self, exc_type, exc_value, traceback):
|
||||||
|
return
|
||||||
Reference in New Issue
Block a user