0

0

基于Paddle2.0实现视频分类模型GSM

P粉084495128

P粉084495128

发布时间:2025-07-18 14:35:00

|

530人浏览过

|

来源于php中文网

原创

本文复现了CVPR 2020的GSM视频分类模型,用Paddle2.0在Diving48数据集实现。该模型通过分解3D卷积核实现时空域视频理解,参数量小。项目处理了数据集,将视频转帧、生成训练测试文件,经训练测试,最佳模型在测试集准确率达0.223。

☞☞☞AI 智能聊天, 问答助手, AI 智能搜索, 免费无限量使用 DeepSeek R1 模型☜☜☜

基于paddle2.0实现视频分类模型gsm - php中文网

项目背景

GSM是2020年CVPR上的一篇论文Gate-Shift Networks for Video Action Recognition中提出的视频分类模型。本项目即对其进行复现。

项目简介

本项目使用paddle2.0实现了视频动作分类GSM模型,并在Diving48数据集上进行了训练和验证。

Diving48数据集的划分是按数据集官方网站的划分方法进行划分的。

模型简介

Gate-Shift Networks for Video Action Recognition (GSM) 的核心是对进行时空域视频理解的3D卷积核进行分解的一种方法。该方法和已有方法的比较如下图所示。从图中可以看出,该模型跟其它网络模块相比具有参数量小的优点。

基于Paddle2.0实现视频分类模型GSM - php中文网            

图1 GSM和其它网络结构比较图

模型细节

模型实现细节如下图所示。

基于Paddle2.0实现视频分类模型GSM - php中文网            

图2 GSM实现细节图

GSM模块可以加在已有网络的不同部分,论文作者通过实验得出,将GSM模块加在下图所示的网络分支示例中具有较好的效果,即卷积核尺寸较小的分支中。

基于Paddle2.0实现视频分类模型GSM - php中文网            

图3 GSM的plug-and-play示意图

论文原文:GSM: Gate-Shift Networks for Video Action Recognition

微尔企业网站管理系统1.75 build build 090709
微尔企业网站管理系统1.75 build build 090709

系统功能介绍 1 包含企业网站所必备的功能:企业信息、产品管理、人才招聘、新闻资讯、企业图片、以及视频下载等模块2 由于是从CMS系统的基础上开发而成,因此相对于一些其他的企业网站管理系统,本系统具备更强的可扩展能力,可以胜任从小型工作室到大中型企业网上门户等各种不同规模网站的需求。3 后台管理与模板完全分离,并具备非常灵活的标签技术,可以实现无限制个性化的界面定制4 操作简单,利用已经制作好的模

下载

数据集介绍

本项目使用Diving48数据集进行训练和测试. 数据集官方网站为Diving48数据集

Diving48是跳水竞赛不同动作的细粒度视频数据集。该数据集包含48类不同的跳水标准动作,共18k左右个视频片段。由于不同的跳水动作可能在起跳、腾空和入水的三个不同阶段中的任一个阶段发生不同,需要模型学习视频中的长时间段间的特征关系。动作示例如下图所示:

基于Paddle2.0实现视频分类模型GSM - php中文网            

图4 Diving48动作示意图


       

这48个类别中的每一个都是由三个不同的动作构成的,分别是起跳动作 (dive groups), 空中动作(somersaults and/or twists), 入水动作 (dive positions)。 下图展示了这48个类别的不同动作组成与互相间的关系:

基于Paddle2.0实现视频分类模型GSM - php中文网            

图5 Diving48中不同动作间关系示意图

文件结构

文件名或文件夹名 功能
model 模型
best_model 训练好的模型
result 训练过程中的数据
dataset 数据文件夹
checkpoints_models 模型文件
pretrained_model 预训练模型
extract_frames_diving48.py 从视频中抽帧
process_dataset_diving.py 生成训练与测试的数据文件
dataset.py 自定义数据
datasets_video.py 读取数据
transforms.py 数据增强
train.py 模型训练
test.py 模型测试

模型实现及训练、验证

解压视频数据集

In [ ]
import osif not os.path.exists('dataset/Diving48'):
    os.makedirs('dataset/Diving48')
   
In [ ]
!tar -xzf data/data56350/Diving48_rgb.tar.gz -C dataset/Diving48
   

创建标签文件

仅在预测时使用

In [ ]
import jsonimport numpy as np

label_dir = {}
i = 0with open('data/data56350/Diving48_vocab.json', 'r') as f:
    vocab_list = json.load(f)    
for vocab in vocab_list:
    label_dir['_'.join(vocab)] = i
    i += 1np.save('work/label_dir.npy', label_dir)print(label_dir)
       
{'Back_15som_05Twis_FREE': 0, 'Back_15som_15Twis_FREE': 1, 'Back_15som_25Twis_FREE': 2, 'Back_15som_NoTwis_PIKE': 3, 'Back_15som_NoTwis_TUCK': 4, 'Back_25som_15Twis_PIKE': 5, 'Back_25som_25Twis_PIKE': 6, 'Back_25som_NoTwis_PIKE': 7, 'Back_25som_NoTwis_TUCK': 8, 'Back_2som_15Twis_FREE': 9, 'Back_2som_25Twis_FREE': 10, 'Back_35som_NoTwis_PIKE': 11, 'Back_35som_NoTwis_TUCK': 12, 'Back_3som_NoTwis_PIKE': 13, 'Back_3som_NoTwis_TUCK': 14, 'Back_Dive_NoTwis_PIKE': 15, 'Back_Dive_NoTwis_TUCK': 16, 'Forward_15som_1Twis_FREE': 17, 'Forward_15som_2Twis_FREE': 18, 'Forward_15som_NoTwis_PIKE': 19, 'Forward_1som_NoTwis_PIKE': 20, 'Forward_25som_1Twis_PIKE': 21, 'Forward_25som_2Twis_PIKE': 22, 'Forward_25som_3Twis_PIKE': 23, 'Forward_25som_NoTwis_PIKE': 24, 'Forward_25som_NoTwis_TUCK': 25, 'Forward_35som_NoTwis_PIKE': 26, 'Forward_35som_NoTwis_TUCK': 27, 'Forward_45som_NoTwis_TUCK': 28, 'Forward_Dive_NoTwis_PIKE': 29, 'Forward_Dive_NoTwis_STR': 30, 'Inward_15som_NoTwis_PIKE': 31, 'Inward_15som_NoTwis_TUCK': 32, 'Inward_25som_NoTwis_PIKE': 33, 'Inward_25som_NoTwis_TUCK': 34, 'Inward_35som_NoTwis_TUCK': 35, 'Inward_Dive_NoTwis_PIKE': 36, 'Reverse_15som_05Twis_FREE': 37, 'Reverse_15som_15Twis_FREE': 38, 'Reverse_15som_25Twis_FREE': 39, 'Reverse_15som_35Twis_FREE': 40, 'Reverse_15som_NoTwis_PIKE': 41, 'Reverse_25som_15Twis_PIKE': 42, 'Reverse_25som_NoTwis_PIKE': 43, 'Reverse_25som_NoTwis_TUCK': 44, 'Reverse_35som_NoTwis_TUCK': 45, 'Reverse_Dive_NoTwis_PIKE': 46, 'Reverse_Dive_NoTwis_TUCK': 47}
       

视频转为jpg格式图片

视频转换需要较长时间,不低于1个小时,建议转换到非data文件夹以便后续启动项目后可以直接使用。

In [10]
!python work/extract_frames_diving48.py
       
ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'dataset/Diving48/rgb/nOlRwoxsDJ0_00555.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.71.100
  Duration: 00:00:03.18, start: 0.000000, bitrate: 838 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 4:3 DAR 16:9], 830 kb/s, 60 fps, 60 tbr, 15360 tbn, 120 tbc (default)
    Metadata:
      handler_name    : VideoHandler[swscaler @ 0x2204380] deprecated pixel format used, make sure you did set range correctlyOutput #0, image2, to 'dataset/Diving48/frames/nOlRwoxsDJ0_00555/frames%05d.jpg':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.40.101
    Stream #0:0(und): Video: mjpeg, yuvj420p(pc), 250x250 [SAR 16:9 DAR 16:9], q=2-31, 200 kb/s, 60 fps, 60 tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc56.60.100 mjpeg
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
frame=  191 fps=0.0 q=24.8 Lsize=N/A time=00:00:03.18 bitrate=N/A    
video:703kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'dataset/Diving48/rgb/OFxuiqI5G44_00406.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.71.100
  Duration: 00:00:03.77, start: 0.000000, bitrate: 1028 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 4:3 DAR 16:9], 1021 kb/s, 60 fps, 60 tbr, 15360 tbn, 120 tbc (default)
    Metadata:
      handler_name    : VideoHandler[swscaler @ 0x18f1400] deprecated pixel format used, make sure you did set range correctlyOutput #0, image2, to 'dataset/Diving48/frames/OFxuiqI5G44_00406/frames%05d.jpg':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.40.101
    Stream #0:0(und): Video: mjpeg, yuvj420p(pc), 250x250 [SAR 16:9 DAR 16:9], q=2-31, 200 kb/s, 60 fps, 60 tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc56.60.100 mjpeg
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
frame=  226 fps=0.0 q=24.8 Lsize=N/A time=00:00:03.76 bitrate=N/A    
video:862kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'dataset/Diving48/rgb/LkluZoNfKu8_00121.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.71.100
  Duration: 00:00:09.40, start: 0.000000, bitrate: 816 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 4:3 DAR 16:9], 814 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler[swscaler @ 0x1b7d3c0] deprecated pixel format used, make sure you did set range correctlyOutput #0, image2, to 'dataset/Diving48/frames/LkluZoNfKu8_00121/frames%05d.jpg':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.40.101
    Stream #0:0(und): Video: mjpeg, yuvj420p(pc), 250x250 [SAR 16:9 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc56.60.100 mjpeg
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
frame=  235 fps=0.0 q=24.8 Lsize=N/A time=00:00:09.40 bitrate=N/A    
video:922kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'dataset/Diving48/rgb/Q--70-ELqLw_00068.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.71.100
  Duration: 00:00:06.56, start: 0.000000, bitrate: 774 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 4:3 DAR 16:9], 771 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler[swscaler @ 0x2108060] deprecated pixel format used, make sure you did set range correctlyOutput #0, image2, to 'dataset/Diving48/frames/Q--70-ELqLw_00068/frames%05d.jpg':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.40.101
    Stream #0:0(und): Video: mjpeg, yuvj420p(pc), 250x250 [SAR 16:9 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc56.60.100 mjpeg
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
frame=  164 fps=0.0 q=24.8 Lsize=N/A time=00:00:06.56 bitrate=N/A    
video:606kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'dataset/Diving48/rgb/_tigfCJFLZg_00205.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.71.100
  Duration: 00:00:04.00, start: 0.000000, bitrate: 1545 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 4:3 DAR 16:9], 1541 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler[swscaler @ 0x1d4e780] deprecated pixel format used, make sure you did set range correctlyOutput #0, image2, to 'dataset/Diving48/frames/_tigfCJFLZg_00205/frames%05d.jpg':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.40.101
    Stream #0:0(und): Video: mjpeg, yuvj420p(pc), 250x250 [SAR 16:9 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc56.60.100 mjpeg
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
frame=  100 fps=0.0 q=24.8 Lsize=N/A time=00:00:04.00 bitrate=N/A    
video:479kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'dataset/Diving48/rgb/sk8TafuB3lU_00696.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.71.100
  Duration: 00:00:01.27, start: 0.000000, bitrate: 1286 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 4:3 DAR 16:9], 1276 kb/s, 60 fps, 60 tbr, 15360 tbn, 120 tbc (default)
    Metadata:
      handler_name    : VideoHandler[swscaler @ 0x1f20b00] deprecated pixel format used, make sure you did set range correctlyOutput #0, image2, to 'dataset/Diving48/frames/sk8TafuB3lU_00696/frames%05d.jpg':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.40.101
    Stream #0:0(und): Video: mjpeg, yuvj420p(pc), 250x250 [SAR 16:9 DAR 16:9], q=2-31, 200 kb/s, 60 fps, 60 tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc56.60.100 mjpeg
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
frame=   76 fps=0.0 q=24.8 Lsize=N/A time=00:00:01.26 bitrate=N/A    
video:199kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'dataset/Diving48/rgb/xbQCwTHcGN8_00158.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.71.100
  Duration: 00:00:03.56, start: 0.000000, bitrate: 1580 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 4:3 DAR 16:9], 1576 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler[swscaler @ 0x1bbd3c0] deprecated pixel format used, make sure you did set range correctlyOutput #0, image2, to 'dataset/Diving48/frames/xbQCwTHcGN8_00158/frames%05d.jpg':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.40.101
    Stream #0:0(und): Video: mjpeg, yuvj420p(pc), 250x250 [SAR 16:9 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc56.60.100 mjpeg
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
frame=   84 fps=0.0 q=24.8 Lsize=N/A time=00:00:03.36 bitrate=N/A    
video:406kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Exiting normally, received signal 2.
ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'dataset/Diving48/rgb/l6ZoMjD1UtA_00156.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.71.100
  Duration: 00:00:05.60, start: 0.000000, bitrate: 908 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 4:3 DAR 16:9], 905 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler[swscaler @ 0x209c500] deprecated pixel format used, make sure you did set range correctlyOutput #0, image2, to 'dataset/Diving48/frames/l6ZoMjD1UtA_00156/frames%05d.jpg':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.40.101
    Stream #0:0(und): Video: mjpeg, yuvj420p(pc), 250x250 [SAR 16:9 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc56.60.100 mjpeg
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
       

生成训练和测试的数据文本文件

In [ ]
!python work/process_dataset_diving.py
       
15026/15027
       

在之前训练基础上微调

In [ ]
!python work/train.py --epoch 5 --batch_size 32 --seg_num 16 --seg_len 1 --pretrain True --weights 'pretrained/inception_pretrained' \
--data_dir 'result/best_model.npz' --best_model_save 'best_model/best_model' --model_arc 'inception'
       
You are using Paddle compiled with TensorRT, but TensorRT dynamic library is not found. Ignore this if TensorRT is not needed.
W0119 22:07:28.029369 15355 device_context.cc:320] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1
W0119 22:07:28.034546 15355 device_context.cc:330] device: 0, cuDNN Version: 7.6.
video number:15027
video number:1970
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/nn/layer/norm.py:634: UserWarning: When training, we now always track global mean and variance.
  "When training, we now always track global mean and variance.")
Loss at epoch 0 step 0: [3.657299], acc: [0.25]
Loss at epoch 0 step 200: [3.6588974], acc: [0.25]
Loss at epoch 0 step 400: [3.6196995], acc: [0.28125]
The best result is 0.22030456
Loss at epoch 1 step 0: [3.5962734], acc: [0.3125]
Loss at epoch 1 step 200: [3.5641773], acc: [0.34375]
Loss at epoch 1 step 400: [3.6840277], acc: [0.21875]
The best result is 0.22081219
Loss at epoch 2 step 0: [3.4678974], acc: [0.4375]
Loss at epoch 2 step 200: [3.4990504], acc: [0.40625]
Loss at epoch 2 step 400: [3.622469], acc: [0.28125]
The best result is 0.22335026
Loss at epoch 3 step 0: [3.6552052], acc: [0.25]
Loss at epoch 3 step 200: [3.5325003], acc: [0.375]
Loss at epoch 3 step 400: [3.6104255], acc: [0.28125]
Loss at epoch 4 step 0: [3.6843047], acc: [0.21875]
Loss at epoch 4 step 200: [3.6228292], acc: [0.28125]
Loss at epoch 4 step 400: [3.7475114], acc: [0.15625]
Final loss: [3.6923075]
time duration is  5594.123642206192
       

图示训练和验证过程

In [ ]
## 画图函数import numpy as npimport matplotlib.pyplot as pltdef draw_train_eval_process(title,step,train_reward, eval_reward, label_train_reward, label_eval_reward, color_train, color_eval, label_y):
    plt.title(title, fontsize=24)
    plt.xlabel("epochs", fontsize=20)
    plt.ylabel(label_y, fontsize=20)
    plt.plot(step, train_reward,color=color_train,label=label_train_reward) 
    plt.plot(step, eval_reward,color=color_eval,label=label_eval_reward) 
    plt.legend()
    plt.grid()
    plt.show()
   
In [ ]
result_gsm = np.load('result/best_model.npz')
gsm_train_rewards = result_gsm['all_train_rewards']
gsm_eval_rewards = result_gsm['all_test_rewards']
steps = [i for i in range(len(gsm_train_rewards))]
draw_train_eval_process("GSM training and eval",steps,gsm_train_rewards,gsm_eval_rewards, 
"GSM trainning accuracy", "GSM eval accuracy", "red","green", "gsm_train_eval_accuracy")
       
               

用保存的最佳训练模型测试

In [ ]
!python work/test.py --weights 'best_model/best_model'
       
You are using Paddle compiled with TensorRT, but TensorRT dynamic library is not found. Ignore this if TensorRT is not needed.
[INFO: test.py:  132]: Namespace(batch_size=1, dataset='diving48', infer_topk=1, modality='RGB', model_arc='inception', seg_len=1, seg_num=16, weights='best_model/best_model')
W0120 00:08:33.052891 25454 device_context.cc:320] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1
W0120 00:08:33.058135 25454 device_context.cc:330] device: 0, cuDNN Version: 7.6.
video number:1970
测试集准确率为:0.22335025668144226
       

热门AI工具

更多
DeepSeek
DeepSeek

幻方量化公司旗下的开源大模型平台

豆包大模型
豆包大模型

字节跳动自主研发的一系列大型语言模型

通义千问
通义千问

阿里巴巴推出的全能AI助手

腾讯元宝
腾讯元宝

腾讯混元平台推出的AI助手

文心一言
文心一言

文心一言是百度开发的AI聊天机器人,通过对话可以生成各种形式的内容。

讯飞写作
讯飞写作

基于讯飞星火大模型的AI写作工具,可以快速生成新闻稿件、品宣文案、工作总结、心得体会等各种文文稿

即梦AI
即梦AI

一站式AI创作平台,免费AI图片和视频生成。

ChatGPT
ChatGPT

最最强大的AI聊天机器人程序,ChatGPT不单是聊天机器人,还能进行撰写邮件、视频脚本、文案、翻译、代码等任务。

相关专题

更多
Python 自然语言处理(NLP)基础与实战
Python 自然语言处理(NLP)基础与实战

本专题系统讲解 Python 在自然语言处理(NLP)领域的基础方法与实战应用,涵盖文本预处理(分词、去停用词)、词性标注、命名实体识别、关键词提取、情感分析,以及常用 NLP 库(NLTK、spaCy)的核心用法。通过真实文本案例,帮助学习者掌握 使用 Python 进行文本分析与语言数据处理的完整流程,适用于内容分析、舆情监测与智能文本应用场景。

22

2026.01.27

拼多多赚钱的5种方法 拼多多赚钱的5种方法
拼多多赚钱的5种方法 拼多多赚钱的5种方法

在拼多多上赚钱主要可以通过无货源模式一件代发、精细化运营特色店铺、参与官方高流量活动、利用拼团机制社交裂变,以及成为多多进宝推广员这5种方法实现。核心策略在于通过低成本、高效率的供应链管理与营销,利用平台社交电商红利实现盈利。

119

2026.01.26

edge浏览器怎样设置主页 edge浏览器自定义设置教程
edge浏览器怎样设置主页 edge浏览器自定义设置教程

在Edge浏览器中设置主页,请依次点击右上角“...”图标 > 设置 > 开始、主页和新建标签页。在“Microsoft Edge 启动时”选择“打开以下页面”,点击“添加新页面”并输入网址。若要使用主页按钮,需在“外观”设置中开启“显示主页按钮”并设定网址。

48

2026.01.26

苹果官方查询网站 苹果手机正品激活查询入口
苹果官方查询网站 苹果手机正品激活查询入口

苹果官方查询网站主要通过 checkcoverage.apple.com/cn/zh/ 进行,可用于查询序列号(SN)对应的保修状态、激活日期及技术支持服务。此外,查找丢失设备请使用 iCloud.com/find,购买信息与物流可访问 Apple (中国大陆) 订单状态页面。

184

2026.01.26

npd人格什么意思 npd人格有什么特征
npd人格什么意思 npd人格有什么特征

NPD(Narcissistic Personality Disorder)即自恋型人格障碍,是一种心理健康问题,特点是极度夸大自我重要性、需要过度赞美与关注,同时极度缺乏共情能力,背后常掩藏着低自尊和不安全感,影响人际关系、工作和生活,通常在青少年时期开始显现,需由专业人士诊断。

7

2026.01.26

windows安全中心怎么关闭 windows安全中心怎么执行操作
windows安全中心怎么关闭 windows安全中心怎么执行操作

关闭Windows安全中心(Windows Defender)可通过系统设置暂时关闭,或使用组策略/注册表永久关闭。最简单的方法是:进入设置 > 隐私和安全性 > Windows安全中心 > 病毒和威胁防护 > 管理设置,将实时保护等选项关闭。

7

2026.01.26

2026年春运抢票攻略大全 春运抢票攻略教你三招手【技巧】
2026年春运抢票攻略大全 春运抢票攻略教你三招手【技巧】

铁路12306提供起售时间查询、起售提醒、购票预填、候补购票及误购限时免费退票五项服务,并强调官方渠道唯一性与信息安全。

178

2026.01.26

个人所得税税率表2026 个人所得税率最新税率表
个人所得税税率表2026 个人所得税率最新税率表

以工资薪金所得为例,应纳税额 = 应纳税所得额 × 税率 - 速算扣除数。应纳税所得额 = 月度收入 - 5000 元 - 专项扣除 - 专项附加扣除 - 依法确定的其他扣除。假设某员工月工资 10000 元,专项扣除 1000 元,专项附加扣除 2000 元,当月应纳税所得额为 10000 - 5000 - 1000 - 2000 = 2000 元,对应税率为 3%,速算扣除数为 0,则当月应纳税额为 2000×3% = 60 元。

39

2026.01.26

oppo云服务官网登录入口 oppo云服务登录手机版
oppo云服务官网登录入口 oppo云服务登录手机版

oppo云服务https://cloud.oppo.com/可以在云端安全存储您的照片、视频、联系人、便签等重要数据。当您的手机数据意外丢失或者需要更换手机时,可以随时将这些存储在云端的数据快速恢复到手机中。

172

2026.01.26

热门下载

更多
网站特效
/
网站源码
/
网站素材
/
前端模板

精品课程

更多
相关推荐
/
热门推荐
/
最新课程
PostgreSQL 教程
PostgreSQL 教程

共48课时 | 7.9万人学习

Git 教程
Git 教程

共21课时 | 3.1万人学习

关于我们 免责申明 举报中心 意见反馈 讲师合作 广告合作 最新更新
php中文网:公益在线php培训,帮助PHP学习者快速成长!
关注服务号 技术交流群
PHP中文网订阅号
每天精选资源文章推送

Copyright 2014-2026 https://www.php.cn/ All Rights Reserved | php.cn | 湘ICP备2023035733号