编译MediaPipe框架
编译MediaPipe框架。
最近开始研究自己的大创项目,一个关于动作捕捉的小玩意儿,第一步就是抄袭开源代码借鉴他人优秀成果。在众多的项目中,我看上了这个Google开源的优秀框架,先把这个项目在本地上跑起来再说。这篇文章就记录了我编译这个框架的过程。
在我写完这篇文章之后,我就从
WSL
润到了Arch Linux
,然而我还是有编译MediaPipe
的需求,所以这篇文章就增加了Arch Linux
下编译MediaPipe
的过程。
首先是在Arch Linux
下编译的过程。
编译环境概述
使用Arch Linux
,需要注意的是Arch Linux
采用滚动更新,目前的安装方法可能在不久的将来就不适用了。
编译
环境准备
首先安装bazelisk
,由于我安装了pnpm
,直接使用pnpm install -g @bazel/bazelisk
安装。
然后使用pacman
安装:
sudo pacman -S opencv ffmpeg jdk-openjdk git
还有编译的过程中可能会涉及到python
中的numpy
软件包,我由于已经安装了conda
来管理python
环境,于是就采用conda install numpy
在base
环境中安装。
克隆MediaPipe
仓库:
git clone https://github.com/google/mediapipe.git
由于pacman
仓库中安装的opencv
版本是最新的opencv4
,我们需要修改MediaPipe
中的配置文件来适配,修改third_party/opencv_linux.BUILD
:
# Description:
# OpenCV libraries for video/image processing on Linux
licenses(["notice"]) # BSD license
exports_files(["LICENSE"])
# The following build rule assumes that OpenCV is installed by
# 'apt-get install libopencv-core-dev libopencv-highgui-dev \'
# ' libopencv-calib3d-dev libopencv-features2d-dev \'
# ' libopencv-imgproc-dev libopencv-video-dev'
# on Debian Buster/Ubuntu 18.04.
# If you install OpenCV separately, please modify the build rule accordingly.
cc_library(
name = "opencv",
hdrs = glob([
# For OpenCV 4.x
#"include/aarch64-linux-gnu/opencv4/opencv2/cvconfig.h",
#"include/arm-linux-gnueabihf/opencv4/opencv2/cvconfig.h",
#"include/x86_64-linux-gnu/opencv4/opencv2/cvconfig.h",
# 将下面这行取消注释
"include/opencv4/opencv2/**/*.h*",
]),
includes = [
# For OpenCV 4.x
#"include/aarch64-linux-gnu/opencv4/",
#"include/arm-linux-gnueabihf/opencv4/",
#"include/x86_64-linux-gnu/opencv4/",
# 将下面这行取消注释
"include/opencv4/",
],
linkopts = [
"-l:libopencv_core.so",
"-l:libopencv_calib3d.so",
"-l:libopencv_features2d.so",
"-l:libopencv_highgui.so",
"-l:libopencv_imgcodecs.so",
"-l:libopencv_imgproc.so",
"-l:libopencv_video.so",
"-l:libopencv_videoio.so",
],
visibility = ["//visibility:public"],
)
编译
跑一下实例中的Hello, World
。
首先设置一个环境变量:
$ export GLOG_logtostderr=1
然后一把梭哈:
bazel run --define MEDIAPIPE_DISABLE_GPU=1 \
mediapipe/examples/desktop/hello_world:hello_world
在第一次编译的时候会下载大量的依赖文件,如果遇到网络错误可以多试几次,我试了三次才完成。
Starting local Bazel server and connecting to it...
DEBUG: /home/ricardo/.cache/bazel/_bazel_ricardo/9d4c3ea39592a9aa5075148d2a9caf3e/external/org_tensorflow/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'com_google_absl' because it already exists.
DEBUG: /home/ricardo/.cache/bazel/_bazel_ricardo/9d4c3ea39592a9aa5075148d2a9caf3e/external/org_tensorflow/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'com_google_benchmark' because it already exists.
DEBUG: /home/ricardo/.cache/bazel/_bazel_ricardo/9d4c3ea39592a9aa5075148d2a9caf3e/external/org_tensorflow/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'flatbuffers' because it already exists.
DEBUG: /home/ricardo/.cache/bazel/_bazel_ricardo/9d4c3ea39592a9aa5075148d2a9caf3e/external/org_tensorflow/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'pybind11_bazel' because it already exists.
DEBUG: /home/ricardo/.cache/bazel/_bazel_ricardo/9d4c3ea39592a9aa5075148d2a9caf3e/external/org_tensorflow/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'com_googlesource_code_re2' because it already exists.
DEBUG: /home/ricardo/.cache/bazel/_bazel_ricardo/9d4c3ea39592a9aa5075148d2a9caf3e/external/org_tensorflow/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'com_google_protobuf' because it already exists.
DEBUG: /home/ricardo/.cache/bazel/_bazel_ricardo/9d4c3ea39592a9aa5075148d2a9caf3e/external/org_tensorflow/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'com_google_googletest' because it already exists.
DEBUG: /home/ricardo/.cache/bazel/_bazel_ricardo/9d4c3ea39592a9aa5075148d2a9caf3e/external/org_tensorflow/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'com_github_gflags_gflags' because it already exists.
DEBUG: /home/ricardo/.cache/bazel/_bazel_ricardo/9d4c3ea39592a9aa5075148d2a9caf3e/external/org_tensorflow/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'zlib' because it already exists.
DEBUG: /home/ricardo/.cache/bazel/_bazel_ricardo/9d4c3ea39592a9aa5075148d2a9caf3e/external/org_tensorflow/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'rules_python' because it already exists.
DEBUG: /home/ricardo/.cache/bazel/_bazel_ricardo/9d4c3ea39592a9aa5075148d2a9caf3e/external/org_tensorflow/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'build_bazel_rules_apple' because it already exists.
DEBUG: /home/ricardo/.cache/bazel/_bazel_ricardo/9d4c3ea39592a9aa5075148d2a9caf3e/external/org_tensorflow/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'build_bazel_rules_swift' because it already exists.
DEBUG: /home/ricardo/.cache/bazel/_bazel_ricardo/9d4c3ea39592a9aa5075148d2a9caf3e/external/org_tensorflow/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'build_bazel_apple_support' because it already exists.
DEBUG: /home/ricardo/.cache/bazel/_bazel_ricardo/9d4c3ea39592a9aa5075148d2a9caf3e/external/org_tensorflow/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'xctestrunner' because it already exists.
DEBUG: /home/ricardo/.cache/bazel/_bazel_ricardo/9d4c3ea39592a9aa5075148d2a9caf3e/external/org_tensorflow/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'pybind11' because it already exists.
WARNING: /home/ricardo/Documents/code/cpp/mediapipe/mediapipe/framework/tool/BUILD:184:24: in cc_library rule //mediapipe/framework/tool:field_data_cc_proto: target '//mediapipe/framework/tool:field_data_cc_proto' depends on deprecated target '@com_google_protobuf//:cc_wkt_protos': Only for backward compatibility. Do not use.
WARNING: /home/ricardo/Documents/code/cpp/mediapipe/mediapipe/framework/BUILD:54:24: in cc_library rule //mediapipe/framework:calculator_cc_proto: target '//mediapipe/framework:calculator_cc_proto' depends on deprecated target '@com_google_protobuf//:cc_wkt_protos': Only for backward compatibility. Do not use.
INFO: Analyzed target //mediapipe/examples/desktop/hello_world:hello_world (84 packages loaded, 1747 targets configured).
INFO: Found 1 target...
Target //mediapipe/examples/desktop/hello_world:hello_world up-to-date:
bazel-bin/mediapipe/examples/desktop/hello_world/hello_world
INFO: Elapsed time: 3.962s, Critical Path: 0.31s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
I20230115 20:26:29.880736 47247 hello_world.cc:57] Hello World!
I20230115 20:26:29.880805 47247 hello_world.cc:57] Hello World!
I20230115 20:26:29.880817 47247 hello_world.cc:57] Hello World!
I20230115 20:26:29.880888 47247 hello_world.cc:57] Hello World!
I20230115 20:26:29.880957 47247 hello_world.cc:57] Hello World!
I20230115 20:26:29.881028 47247 hello_world.cc:57] Hello World!
I20230115 20:26:29.881096 47247 hello_world.cc:57] Hello World!
I20230115 20:26:29.881157 47247 hello_world.cc:57] Hello World!
I20230115 20:26:29.881198 47247 hello_world.cc:57] Hello World!
I20230115 20:26:29.881268 47247 hello_world.cc:57] Hello World!
编译Pose解决方案的Android Archieve
我的需求有一个是在安卓手机上使用MediaPipe
的Pose
解决方案,需要编译出一个AAR
安卓打包文件。
首先安装需要的依赖,Android SDK
和Android NDK
。这里使用Android Studio
安装。
然后创建一个bazel
编译的目标文件:在目录mediapipe/examples/android/src/java/com/google/mediapipe/apps/
下创建一个文件夹pose_tracking_aar
,在其中创建BUILD
文件,写入:
load("//mediapipe/java/com/google/mediapipe:mediapipe_aar.bzl", "mediapipe_aar")
mediapipe_aar(
name = "mediapipe_pose_tracking",
calculators = ["//mediapipe/graphs/pose_tracking:pose_tracking_gpu_deps"],
)
使用命令编译:
bazel build -c opt --strip=ALWAYS \
--host_crosstool_top=@bazel_tools//tools/cpp:toolchain \
--fat_apk_cpu=arm64-v8a,armeabi-v7a \
--legacy_whole_archive=0 \
--features=-legacy_whole_archive \
--copt=-fvisibility=hidden \
--copt=-ffunction-sections \
--copt=-fdata-sections \
--copt=-fstack-protector \
--copt=-Oz \
--copt=-fomit-frame-pointer \
--copt=-DABSL_MIN_LOG_LEVEL=2 \
--linkopt=-Wl,--gc-sections,--strip-all \
//mediapipe/examples/android/src/java/com/google/mediapipe/apps/pose_tracking_aar:mediapipe_pose_tracking.aar
如果在编译的过程中提示缺失dx.jar
这个文件而且你用的SDK版本还是高于31的,那可能是SDK中缺失了这个文件,可以将SDk降级到30就含有这个文件了。我使用的解决办法比较离奇,我是将30版本的SDK文件中的这个文件软链接过来,解决了这个问题。
编译消耗的时间可能比较的长,耐心等待即可。
为了在手机上使用,我们还需要编译出binarypb
文件,从Google的服务器上下载tflite
文件。
编译binarypb
的过程比较的简单,编译目标在mediapipe/graphs/pose_tracking
中,名称是pose_tracking_gpu_binary_graph
,使用下列指令编译:
bazel build -c opt //mediapipe/graphs/pose_tracking:pose_tracking_gpu_binary_graph
在这里,Google默认添加了一个
input side packet
打开人体遮罩,如果不需要这个效果,需要删除mediapipe/graphs/pose_tracking/pose_tracking_gpu.pbtxt
文件中的以下内容:# Generates side packet to enable segmentation. node { calculator: "ConstantSidePacketCalculator" output_side_packet: "PACKET:enable_segmentation" node_options: { [type.googleapis.com/mediapipe.ConstantSidePacketCalculatorOptions]: { packet { bool_value: true } } } }
然后还需要从服务器上下载tflite
文件,Pose Tracking
这个解决方案需要两个tflite
文件,第一个是pose_detection.tflite,第二个文件则有三个不同的选择,分别对于解决方案中提供的三个质量版本:
下载地址是pose_landmark_full.tflite,pose_landmark_heavy.tflite和pose_landmark_lite.tflite。
下面是原来使用
WSL
编译的过程。
编译环境概述
我使用的基于WSL2的Ubuntu 22.04编译。主要是参考官方的安装文档。
编译
环境准备
首先安装两个基础性的包:
sudo apt install build-essential git
然后安装MediaPipe的编译管理工具bazel
,这里我是通过npm
安装:
pnpm add -g @bazel/bazelisk
通过运行bazel version
验证安装成功:
$ bazel version
Bazelisk version: v1.13.2
WARNING: Invoking Bazel in batch mode since it is not invoked from within a workspace (below a directory having a WORKSPACE file).
Extracting Bazel installation...
Build label: 5.3.2
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Wed Oct 19 18:22:12 2022 (1666203732)
Build timestamp: 1666203732
Build timestamp as int: 1666203732
安装Miniconda
,再在环境中安装numpy
。编译中依赖于Python
和numpy
,这里网上的资料汗牛充栋,我就不过多赘述。
在准备玩上面的环境之后,我们就可以用git
将MediaPipe的源代码克隆下来了。
git clone https://github.com/google/mediapipe.git
安装Opencv和FFmpeg
我这里选择的是手动编译安装opencv,安装的步骤参考官方的安装脚本,但是脚本中的不少内容已经过时。
首先安装必要的依赖库和编译管理工具:
sudo apt install cmake ffmpeg libavformat-dev libdc1394-dev libgtk2.0-dev \
libjpeg-dev libpng-dev libswscale-dev libtbb2 libtbb-dev \
libtiff-dev
注意:官方脚本中要求安装
libdc1394-22-dev
这个包,但是按照这篇回答,这个包已经被libdc1392-dev
取代了。
在临时文件夹中创建一个文件专门用来编译:
cd /tmp
mkdir opencv
cd opencv/
使用git
下载Opencv的源代码:
git clone https://github.com/opencv/opencv_contrib.git
git clone https://github.com/opencv/opencv.git
在仓库中签出到指定的版本分支:
cd opencv
git checkout 3.4
cd ../opencv_contrib
git checkout 3.4
创建编译文件,使用指定的cmake
参数生成编译文件:
cd ../opencv
mkdir release
cd release
cmake .. -DCMAKE_BUILD_TYPE=RELEASE -DCMAKE_INSTALL_PREFIX=/usr/local \
-DBUILD_TESTS=OFF -DBUILD_PERF_TESTS=OFF -DBUILD_opencv_ts=OFF \
-DOPENCV_EXTRA_MODULES_PATH=/tmp/opencv/opencv_contrib/modules \
-DBUILD_opencv_aruco=OFF -DBUILD_opencv_bgsegm=OFF -DBUILD_opencv_bioinspired=OFF \
-DBUILD_opencv_ccalib=OFF -DBUILD_opencv_datasets=OFF -DBUILD_opencv_dnn=OFF \
-DBUILD_opencv_dnn_objdetect=OFF -DBUILD_opencv_dpm=OFF -DBUILD_opencv_face=OFF \
-DBUILD_opencv_fuzzy=OFF -DBUILD_opencv_hfs=OFF -DBUILD_opencv_img_hash=OFF \
-DBUILD_opencv_js=OFF -DBUILD_opencv_line_descriptor=OFF -DBUILD_opencv_phase_unwrapping=OFF \
-DBUILD_opencv_plot=OFF -DBUILD_opencv_quality=OFF -DBUILD_opencv_reg=OFF \
-DBUILD_opencv_rgbd=OFF -DBUILD_opencv_saliency=OFF -DBUILD_opencv_shape=OFF \
-DBUILD_opencv_structured_light=OFF -DBUILD_opencv_surface_matching=OFF \
-DBUILD_opencv_world=OFF -DBUILD_opencv_xobjdetect=OFF -DBUILD_opencv_xphoto=OFF \
-DCV_ENABLE_INTRINSICS=ON -DWITH_EIGEN=ON -DWITH_PTHREADS=ON -DWITH_PTHREADS_PF=ON \
-DWITH_JPEG=ON -DWITH_PNG=ON -DWITH_TIFF=ON
注意安装自己下载源代码的地址修改
-DOPENCV_EXTRA_MUDULES_PATH
的值
安装过程中还会下载一系列的依赖包,请注意自己的网络环境
使用make
指令进行编译和安装
make -j 16
sudo make install
编辑链接器的配置:
sudo touch /etc/ld.so.conf.d/mp_opencv.conf
sudo bash -c "echo /usr/local/lib >> /etc/ld.so.conf.d/mp_opencv.conf"
sudo ldconfig -v
然后进行MediaPipe的目录,用脚本进行配置文件的修改:
./setup_opencv.sh config_only
运行首个例子:
export GLOG_logtostderr=1
bazel run --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/hello_world:hello_world
在等待一段时间的下载依赖和编译之后,我们可以看见:
I20221110 22:00:50.899885 14357 hello_world.cc:57] Hello World!
I20221110 22:00:50.899948 14357 hello_world.cc:57] Hello World!
I20221110 22:00:50.899955 14357 hello_world.cc:57] Hello World!
I20221110 22:00:50.899960 14357 hello_world.cc:57] Hello World!
I20221110 22:00:50.899962 14357 hello_world.cc:57] Hello World!
I20221110 22:00:50.899982 14357 hello_world.cc:57] Hello World!
I20221110 22:00:50.900000 14357 hello_world.cc:57] Hello World!
I20221110 22:00:50.900025 14357 hello_world.cc:57] Hello World!
I20221110 22:00:50.900030 14357 hello_world.cc:57] Hello World!
I20221110 22:00:50.900193 14357 hello_world.cc:57] Hello World!
如果出现了各种奇怪的报错,那可以执行这条命令重新安装依赖再编译试试:
bazel clean --expunge
2021 - 2025 © Ricardo Ren, 由 .NET 9.0.1 驱动。