本文介绍基于paddlehub实现一键视频动漫化的方法。先升级安装paddlehub，设置gpu环境，再通过代码将视频提取为图片，选用animegan_v2系列模型（如宫崎骏、新海诚风格）将图片转为动漫风格并合成视频，最后添加原始音频，完成视频动漫化转换。
☞☞☞AI 智能聊天, 问答助手, AI 智能搜索, 免费无限量使用 DeepSeek R1 模型☜☜☜
PaddleHub一键视频动漫化
引子
作为日本动漫的爱好者，特别是对宫崎骏、新海诚的童鞋们有福了。
本项目基于 PaddleHub 的动漫风格的转换模型，可以一键将视频转动漫哦！——>{日系风Vlog就在这}。
PaddleHub已将定制打造的街景动漫化模型animegan_v2_hayao_99、animegan_v2_shinkai_53、animegan_v2_paprika_98等多个优质模型开源。可以自行切换转换风格！
效果展示图
动漫化结果
一、安装PaddleHub
In [1]
#由于PaddleHub升级比较快，建议大家直接升级到最新版本的PaddleHub!pip install --upgrade paddlehub -i https://pypi.tuna.tsinghua.edu.cn/simple
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Requirement already up-to-date: paddlehub in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (2.0.4) Requirement already satisfied, skipping upgrade: packaging in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (20.9) Requirement already satisfied, skipping upgrade: pyzmq in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (18.1.1) Requirement already satisfied, skipping upgrade: colorama in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (0.4.4) Requirement already satisfied, skipping upgrade: filelock in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (3.0.12) Requirement already satisfied, skipping upgrade: gitpython in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (3.1.14) Requirement already satisfied, skipping upgrade: numpy in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (1.16.4) Requirement already satisfied, skipping upgrade: matplotlib in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (2.2.3) Requirement already satisfied, skipping upgrade: tqdm in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (4.36.1) Requirement already satisfied, skipping upgrade: gunicorn>=19.10.0; sys_platform != "win32" in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (20.0.4) Requirement already satisfied, skipping upgrade: pyyaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (5.1.2) Requirement already satisfied, skipping upgrade: Pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (7.1.2) Requirement already satisfied, skipping upgrade: visualdl>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (2.1.1) Requirement already satisfied, skipping upgrade: paddlenlp>=2.0.0rc5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (2.0.0rc7) Requirement already satisfied, skipping upgrade: opencv-python in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (4.1.1.26) Requirement already satisfied, skipping upgrade: colorlog in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (4.1.0) Requirement already satisfied, skipping upgrade: rarfile in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (3.1) Requirement already satisfied, skipping upgrade: easydict in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (1.9) Requirement already satisfied, skipping upgrade: flask>=1.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub) (1.1.1) Requirement already satisfied, skipping upgrade: pyparsing>=2.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from packaging->paddlehub) (2.4.2) Requirement already satisfied, skipping upgrade: gitdb<5,>=4.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from gitpython->paddlehub) (4.0.5) Requirement already satisfied, skipping upgrade: cycler>=0.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->paddlehub) (0.10.0) Requirement already satisfied, skipping upgrade: pytz in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->paddlehub) (2019.3) Requirement already satisfied, skipping upgrade: python-dateutil>=2.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->paddlehub) (2.8.0) Requirement already satisfied, skipping upgrade: six>=1.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->paddlehub) (1.15.0) Requirement already satisfied, skipping upgrade: kiwisolver>=1.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->paddlehub) (1.1.0) Requirement already satisfied, skipping upgrade: setuptools>=3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from gunicorn>=19.10.0; sys_platform != "win32"->paddlehub) (41.4.0) Requirement already satisfied, skipping upgrade: pre-commit in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.0.0->paddlehub) (1.21.0) Requirement already satisfied, skipping upgrade: Flask-Babel>=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.0.0->paddlehub) (1.0.0) Requirement already satisfied, skipping upgrade: requests in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.0.0->paddlehub) (2.22.0) Requirement already satisfied, skipping upgrade: bce-python-sdk in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.0.0->paddlehub) (0.8.53) Requirement already satisfied, skipping upgrade: shellcheck-py in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.0.0->paddlehub) (0.7.1.1) Requirement already satisfied, skipping upgrade: flake8>=3.7.9 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.0.0->paddlehub) (3.8.2) Requirement already satisfied, skipping upgrade: protobuf>=3.11.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl>=2.0.0->paddlehub) (3.14.0) Requirement already satisfied, skipping upgrade: jieba in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp>=2.0.0rc5->paddlehub) (0.42.1) Requirement already satisfied, skipping upgrade: seqeval in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp>=2.0.0rc5->paddlehub) (1.2.2) Requirement already satisfied, skipping upgrade: h5py in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp>=2.0.0rc5->paddlehub) (2.9.0) Requirement already satisfied, skipping upgrade: Werkzeug>=0.15 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.0->paddlehub) (0.16.0) Requirement already satisfied, skipping upgrade: click>=5.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.0->paddlehub) (7.0) Requirement already satisfied, skipping upgrade: Jinja2>=2.10.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.0->paddlehub) (2.10.1) Requirement already satisfied, skipping upgrade: itsdangerous>=0.24 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.0->paddlehub) (1.1.0) Requirement already satisfied, skipping upgrade: smmap<4,>=3.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from gitdb<5,>=4.0.1->gitpython->paddlehub) (3.0.5) Requirement already satisfied, skipping upgrade: virtualenv>=15.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlehub) (16.7.9) Requirement already satisfied, skipping upgrade: identify>=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlehub) (1.4.10) Requirement already satisfied, skipping upgrade: cfgv>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlehub) (2.0.1) Requirement already satisfied, skipping upgrade: importlib-metadata; python_version < "3.8" in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlehub) (0.23) Requirement already satisfied, skipping upgrade: aspy.yaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlehub) (1.3.0) Requirement already satisfied, skipping upgrade: toml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlehub) (0.10.0) Requirement already satisfied, skipping upgrade: nodeenv>=0.11.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl>=2.0.0->paddlehub) (1.3.4) Requirement already satisfied, skipping upgrade: Babel>=2.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask-Babel>=1.0.0->visualdl>=2.0.0->paddlehub) (2.8.0) Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl>=2.0.0->paddlehub) (2.8) Requirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl>=2.0.0->paddlehub) (1.25.6) Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl>=2.0.0->paddlehub) (3.0.4) Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl>=2.0.0->paddlehub) (2019.9.11) Requirement already satisfied, skipping upgrade: pycryptodome>=3.8.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from bce-python-sdk->visualdl>=2.0.0->paddlehub) (3.9.9) Requirement already satisfied, skipping upgrade: future>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from bce-python-sdk->visualdl>=2.0.0->paddlehub) (0.18.0) Requirement already satisfied, skipping upgrade: pyflakes<2.3.0,>=2.2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl>=2.0.0->paddlehub) (2.2.0) Requirement already satisfied, skipping upgrade: pycodestyle<2.7.0,>=2.6.0a1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl>=2.0.0->paddlehub) (2.6.0) Requirement already satisfied, skipping upgrade: mccabe<0.7.0,>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl>=2.0.0->paddlehub) (0.6.1) Requirement already satisfied, skipping upgrade: scikit-learn>=0.21.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from seqeval->paddlenlp>=2.0.0rc5->paddlehub) (0.22.1) Requirement already satisfied, skipping upgrade: MarkupSafe>=0.23 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Jinja2>=2.10.1->flask>=1.1.0->paddlehub) (1.1.1) Requirement already satisfied, skipping upgrade: zipp>=0.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from importlib-metadata; python_version < "3.8"->pre-commit->visualdl>=2.0.0->paddlehub) (0.6.0) Requirement already satisfied, skipping upgrade: joblib>=0.11 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp>=2.0.0rc5->paddlehub) (0.14.1) Requirement already satisfied, skipping upgrade: scipy>=0.17.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp>=2.0.0rc5->paddlehub) (1.3.0) Requirement already satisfied, skipping upgrade: more-itertools in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from zipp>=0.5->importlib-metadata; python_version < "3.8"->pre-commit->visualdl>=2.0.0->paddlehub) (7.2.0)
In [2]
# 该模型需要gpu跑，所以需要设置下gpu环境%env CUDA_VISIBLE_DEVICES=0%matplotlib inline
env: CUDA_VISIBLE_DEVICES=0
In [3]
import cv2from PIL import Imageimport numpy as npimport osimport paddlehub as hubfrom moviepy.editor import *from tqdm import tqdm# Config# 原始视频地址original_video_path = './1.mp4'# 提取视频图像的存放地址original_video_img_path = './original_video_img/'# 合成视频存放地址img2video_path = './2.mp4'# 添加声音后的视频最终输出地址output_video_path = './3.mp4'# 从视频提取图片def video2img(video_path, out_path):
cap = cv2.VideoCapture(video_path)
i=1
while True:
ret, frame = cap.read() if frame is None: break
else:
cv2.imwrite(out_path + str(i) + ".jpg", frame)
i+=1
return# 把图片转动漫并合成视频def ani2video(img_path, org_video_path, out_path, model):
# 获取图片总数
file_list = os.listdir(img_path)
img_num = len(file_list) # 查看原始视频的参数
cap = cv2.VideoCapture(org_video_path)
ret, frame = cap.read() # 任选一张图片查看高度和宽度
result = model.style_transfer(images=[cv2.imread(os.path.join(img_path,file_list[0]))])
height = result[0].shape[0]
width = result[0].shape[1]
fps = cap.get(cv2.CAP_PROP_FPS) # 返回视频的fps--帧率
# 把参数用到我们要创建的视频上
video = cv2.VideoWriter(out_path, cv2.VideoWriter_fourcc('m', 'p', '4', 'v'), fps, (width, height)) # 创建视频流对象
"""
参数1 即将保存的文件路径
参数2 VideoWriter_fourcc为视频编解码器 cv2.VideoWriter_fourcc('m', 'p', '4', 'v') 文件名后缀为.mp4
参数3 为帧播放速率
参数4 (width,height)为视频帧大小
"""
for i in tqdm(range(img_num)):
f_name = str(i + 1) + '.jpg'
item = os.path.join(img_path, f_name)
result = model.style_transfer(images=[cv2.imread(item)]) # 转换动漫风格
video.write(result[0]) # 把图片写进视频
video.release() # 释放# 从原始视频上提取声音合成到新生成的视频上def sound2video(org_video_path, new_video_path, out_video_path):
# 读取原始视频
video_o = VideoFileClip(org_video_path) # 获取原始视频的音频部分
audio_o = video_o.audio # 读取新生成视频
video_clip = VideoFileClip(new_video_path) # 指向新生成视频的音频部分
video_clip2 = video_clip.set_audio(audio_o) # 修改音频部分并输出最终视频
video_clip2.write_videofile(out_video_path)
二、从视频提取图像
In [4]
# 第一步：视频->图像if not os.path.exists(original_video_img_path):
os.mkdir(original_video_img_path)
video2img(video_path=original_video_path, out_path=original_video_img_path)
三、图像转换为动漫风格并合成新的视频
In [5]
# 第二步：转换为动漫效果并合成视频# 根据自己喜好选择风格：## 今敏:'animegan_v2_paprika_98'## 新海诚:'animegan_v2_shinkai_53'## 宫崎骏:'animegan_v2_hayao_99'model = hub.Module(name='animegan_v2_hayao_99', use_gpu=True) ani2video(img_path=original_video_img_path, org_video_path=original_video_path, out_path=img2video_path, model=model)
2021-03-11 09:46:23,753 - INFO - Lock 139930877856336 acquired on /home/aistudio/.paddlehub/tmp/animegan_v2_hayao_99 [INFO 2021-03-11 09:46:23,753 filelock.py:274] Lock 139930877856336 acquired on /home/aistudio/.paddlehub/tmp/animegan_v2_hayao_99
Download https://bj.bcebos.com/paddlehub/paddlehub_dev/animegan_v2_hayao_99.tar.gz [##################################################] 100.00% Decompress /home/aistudio/.paddlehub/tmp/tmp3vmh4rme/animegan_v2_hayao_99.tar.gz [##################################################] 100.00%
[2021-03-11 09:46:24,520] [ INFO] - Successfully installed animegan_v2_hayao_99-1.0.1 2021-03-11 09:46:24,532 - INFO - Lock 139930877856336 released on /home/aistudio/.paddlehub/tmp/animegan_v2_hayao_99 [INFO 2021-03-11 09:46:24,532 filelock.py:318] Lock 139930877856336 released on /home/aistudio/.paddlehub/tmp/animegan_v2_hayao_99 [2021-03-11 09:46:24,533] [ WARNING] - The _initialize method in HubModule will soon be deprecated, you can use the __init__() to handle the initialization of the object 100%|██████████| 2402/2402 [07:05<00:00, 5.65it/s]
四、添加原始音频
In [6]
# 第三步：加上原始音频if not os.path.exists(output_video_path):
sound2video(org_video_path=original_video_path, new_video_path=img2video_path, out_video_path=output_video_path)else: print('最终视频已存在，请查看输出路径')
Moviepy - Building video ./3.mp4. MoviePy - Writing audio in 3TEMP_MPY_wvf_snd.mp3
<br/>
MoviePy - Done. Moviepy - Writing video ./3.mp4
t: 100%|█████████▉| 2398/2403 [00:34<00:00, 90.09it/s, now=None]2021-03-11 09:54:14,198 - WARNING - /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/moviepy/video/io/ffmpeg_reader.py:130: UserWarning: Warning: in file ./2.mp4, 1769472 bytes wanted but 0 bytes read,at frame 2402/2403, at time 80.15/80.15 sec. Using the last valid frame instead. UserWarning) [WARNING 2021-03-11 09:54:14,198 warnings.py:110] /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/moviepy/video/io/ffmpeg_reader.py:130: UserWarning: Warning: in file ./2.mp4, 1769472 bytes wanted but 0 bytes read,at frame 2402/2403, at time 80.15/80.15 sec. Using the last valid frame instead. UserWarning)
Moviepy - Done ! Moviepy - video ready ./3.mp4
代码解释
