caffe编译是一个繁琐个过程, 或多或少会遇到各种各样的问题, 如我caffe编译错误记录记录了遇到过得一部分问题. 原因是因为每个依赖如openblas, protobuf都需要编译特定版本, 每台电脑装的依赖库各不相同, 环境变量也有冲突的情况(如system和anaconda). 于是我一直想找一种简单方便的caffe编译方法, 可以避免上述的问题. 也就有了今天的内容, 使用anaconda虚拟环境编译caffe.
一些说明:
- anaconda虚拟环境可以把环境和系统隔离, 相当于浓缩版
/usr/lib+/usr/include, 在环境中安装包使用conda install {包名称}=版本号命令, 可以很方便的修改环境依赖; - 测试系统是Manjaro Linux, 但是使用anaconda+cmake, 应该和系统没有关系. 即使用其他linux系统或者windows也可以;
- anaconda环境名称为caffe_27． python 3在编译完成caffe后, pychaffe编译出错. 于是换成了python2.7. python3我会后续补充;
- 我的caffe是添加了多标签的部分, 即label可以有多个, 可以直接用我修改好的caffe开始编译;
- cuda任然需要本地安装, 我的是Manjaro Linux, 使用
pacman -S cuda安装. - 项目整体只在本地测试过, 如果有问题还请指出, 我会进一步完善.
- NCCL, GPU默认开启，如果需要关闭可以更改 CmakeList.txt相关位置．
(1) 生成anaconda虚拟环境
我编译完成后, 把用到的conda虚拟环境输出成了.yml文件. 重新编译时, 可以使用conda env create -f caffe_36.yml生成和我一模一样的虚拟环境,而不需要一个一个包重新安装.
(2) 激活虚拟环境
环境名我定的名字是caffe_ 27, 代表python版本是2.7(python3.6环境编译可以通过, 但是pycaffe编辑通过, 于是后来换成了python 2.7, ), 用于编译caffe环境. 如果使用第一步命令生成环境, 那么可以运行一下source命令激活.source activate caffe_27
(3) cmake
不使用make, 而是用camke进行编译, 一方面可以跨平台, 另一方面cmake更加通俗易懂.
需要进行如下一些修改:
<1> 修改虚拟环境路径:
<2> 如果cuda有问题, 手动添加cuda路径:
执行如下命令:
mkdir build
cd build
cmake ..
camke过程如图:
-- The C compiler identification is GNU 8.3.0
-- The CXX compiler identification is GNU 8.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Warning (dev) at cmake/Misc.cmake:32 (set):
implicitly converting 'BOOLEAN' to 'STRING' type.
Call Stack (most recent call first):
CMakeLists.txt:29 (include)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found CUDA: /opt/cuda (found version "10.1")
-- Boost version: 1.60.0
-- Found the following Boost libraries:
-- system
-- thread
-- filesystem
-- chrono
-- date_time
-- atomic
-- Found GFlags: /home/geoffrey/.conda/envs/caffe_27/include
-- Found gflags (include: /home/geoffrey/.conda/envs/caffe_27/include, library: /home/geoffrey/.conda/envs/caffe_27/lib/libgflags.so)
-- Found Glog: /home/geoffrey/.conda/envs/caffe_27/include
-- Found glog (include: /home/geoffrey/.conda/envs/caffe_27/include, library: /home/geoffrey/.conda/envs/caffe_27/lib/libglog.so)
-- Found Protobuf: /home/geoffrey/.conda/envs/caffe_27/lib/libprotobuf-lite.so;/home/geoffrey/.conda/envs/caffe_27/lib/libprotoc.so;/home/geoffrey/.conda/envs/caffe_27/lib/libprotobuf.so (found version "2.5.0")
-- Found PROTOBUF Compiler: /home/geoffrey/.conda/envs/caffe_27/bin/protoc
-- HDF5: Using hdf5 compiler wrapper to determine C configuration
-- HDF5: Using hdf5 compiler wrapper to determine CXX configuration
-- Found HDF5: /home/geoffrey/.conda/envs/caffe_27/lib/libhdf5_cpp.so;/home/geoffrey/.conda/envs/caffe_27/lib/libhdf5.so;/usr/lib/librt.so;/usr/lib/libpthread.so;/home/geoffrey/.conda/envs/caffe_27/lib/libz.so;/usr/lib/libdl.so;/usr/lib/libm.so (found version "1.10.2") found components: HL
-- Found LMDB: /home/geoffrey/.conda/envs/caffe_27/include
-- Found lmdb (include: /home/geoffrey/.conda/envs/caffe_27/include, library: /home/geoffrey/.conda/envs/caffe_27/lib/liblmdb.so)
-- Found LevelDB: /home/geoffrey/.conda/envs/caffe_27/include
-- Found LevelDB (include: /home/geoffrey/.conda/envs/caffe_27/include, library: /home/geoffrey/.conda/envs/caffe_27/lib/libleveldb.so)
-- Found Snappy: /home/geoffrey/.conda/envs/caffe_27/include
-- Found Snappy (include: /home/geoffrey/.conda/envs/caffe_27/include, library: /home/geoffrey/.conda/envs/caffe_27/lib/libsnappy.so)
-- CUDA detected: 10.1
-- Found cuDNN: ver. 7.6.0 found (include: /usr/include, library: /usr/lib/libcudnn.so)
-- Added CUDA NVCC flags for: sm_61
-- Found NCCL: /home/geoffrey/.conda/envs/caffe_27/include
-- Found NCCL (include: /home/geoffrey/.conda/envs/caffe_27/include, library: /home/geoffrey/.conda/envs/caffe_27/lib/libnccl.so)
-- OpenCV found (/home/geoffrey/.conda/envs/caffe_27/share/OpenCV)
-- Found Atlas: /home/geoffrey/.conda/envs/caffe_27/include
-- Found Atlas (include: /home/geoffrey/.conda/envs/caffe_27/include library: /home/geoffrey/.conda/envs/caffe_27/lib/libblas.so lapack: /home/geoffrey/.conda/envs/caffe_27/lib/liblapack.so
-- Found PythonInterp: /home/geoffrey/.conda/envs/caffe_27/bin/python2.7 (found suitable version "2.7.16", minimum required is "2.7")
-- Found PythonLibs: /home/geoffrey/.conda/envs/caffe_27/lib/libpython2.7.so (found suitable version "2.7.16", minimum required is "2.7")
-- Found NumPy: /home/geoffrey/.conda/envs/caffe_27/lib/python2.7/site-packages/numpy/core/include (found suitable version "1.8.2", minimum required is "1.7.1")
-- NumPy ver. 1.8.2 found (include: /home/geoffrey/.conda/envs/caffe_27/lib/python2.7/site-packages/numpy/core/include)
-- Boost version: 1.60.0
-- Found the following Boost libraries:
-- python
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Found Git: /usr/bin/git (found version "2.22.0")
--
-- ******************* Caffe Configuration Summary *******************
-- General:
-- Version : 1.0.0
-- Git : unknown
-- System : Linux
-- C++ compiler : /usr/bin/c++
-- Release CXX flags : -O3 -DNDEBUG -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
-- Debug CXX flags : -g -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
-- Build type : Release
--
-- BUILD_SHARED_LIBS : ON
-- BUILD_python : ON
-- BUILD_matlab : OFF
-- BUILD_docs : ON
-- CPU_ONLY : OFF
-- USE_OPENCV : ON
-- USE_LEVELDB : ON
-- USE_LMDB : ON
-- USE_NCCL : ON
-- ALLOW_LMDB_NOLOCK : OFF
-- USE_HDF5 : ON
--
-- Dependencies:
-- BLAS : Yes (Atlas)
-- Boost : Yes (ver. 1.60)
-- glog : Yes
-- gflags : Yes
-- protobuf : Yes (ver. 2.5.0)
-- lmdb : Yes (ver. 0.9.23)
-- LevelDB : Yes (ver. 1.19)
-- Snappy : Yes (ver. ..)
-- OpenCV : Yes (ver. 3.4.2)
-- CUDA : Yes (ver. 10.1)
--
-- NVIDIA CUDA:
-- Target GPU(s) : Auto
-- GPU arch(s) : sm_61
-- cuDNN : Yes (ver. 7.6.0)
--
-- Python:
-- Interpreter : /home/geoffrey/.conda/envs/caffe_27/bin/python2.7 (ver. 2.7.16)
-- Libraries : /home/geoffrey/.conda/envs/caffe_27/lib/libpython2.7.so (ver 2.7.16)
-- NumPy : /home/geoffrey/.conda/envs/caffe_27/lib/python2.7/site-packages/numpy/core/include (ver 1.8.2)
--
-- Documentaion:
-- Doxygen : No
-- config_file :
--
-- Install:
-- Install path : /home/geoffrey/caffe/build/install
--
Caffe_INCLUDES: PUBLIC;/home/geoffrey/.conda/envs/caffe_27/include;PUBLIC;/home/geoffrey/.conda/envs/caffe_27/include;PUBLIC;/home/geoffrey/.conda/envs/caffe_27/include;PUBLIC;/home/geoffrey/.conda/envs/caffe_27/include;PUBLIC;/home/geoffrey/.conda/envs/caffe_27/include;PUBLIC;/home/geoffrey/.conda/envs/caffe_27/include;PUBLIC;/home/geoffrey/.conda/envs/caffe_27/include;PRIVATE;/home/geoffrey/.conda/envs/caffe_27/include;PUBLIC;/opt/cuda/include;PUBLIC;/usr/include;PUBLIC;/home/geoffrey/.conda/envs/caffe_27/include;/home/geoffrey/.conda/envs/caffe_27/include/opencv;PUBLIC;/home/geoffrey/.conda/envs/caffe_27/include;/home/geoffrey/.conda/envs/caffe_27/include;PRIVATE;/home/geoffrey/.conda/envs/caffe_27/include/python2.7;/home/geoffrey/.conda/envs/caffe_27/lib/python2.7/site-packages/numpy/core/include;PUBLIC;/home/geoffrey/.conda/envs/caffe_27/include
Caffe_LINKER_LIBS: PUBLIC;/home/geoffrey/.conda/envs/caffe_27/lib/libboost_system.so;/home/geoffrey/.conda/envs/caffe_27/lib/libboost_thread.so;-lpthread;/home/geoffrey/.conda/envs/caffe_27/lib/libboost_filesystem.so;/home/geoffrey/.conda/envs/caffe_27/lib/libboost_chrono.so;/home/geoffrey/.conda/envs/caffe_27/lib/libboost_date_time.so;/home/geoffrey/.conda/envs/caffe_27/lib/libboost_atomic.so;PRIVATE;-lpthread;PUBLIC;/home/geoffrey/.conda/envs/caffe_27/lib/libglog.so;PUBLIC;/home/geoffrey/.conda/envs/caffe_27/lib/libgflags.so;PUBLIC;/home/geoffrey/.conda/envs/caffe_27/lib/libprotobuf-lite.so;/home/geoffrey/.conda/envs/caffe_27/lib/libprotoc.so;/home/geoffrey/.conda/envs/caffe_27/lib/libprotobuf.so;PUBLIC;/home/geoffrey/.conda/envs/caffe_27/lib/libhdf5_cpp.so;/home/geoffrey/.conda/envs/caffe_27/lib/libhdf5.so;/usr/lib/librt.so;/usr/lib/libpthread.so;/home/geoffrey/.conda/envs/caffe_27/lib/libz.so;/usr/lib/libdl.so;/usr/lib/libm.so;/home/geoffrey/.conda/envs/caffe_27/lib/libhdf5_hl_cpp.so;/home/geoffrey/.conda/envs/caffe_27/lib/libhdf5_hl.so;/home/geoffrey/.conda/envs/caffe_27/lib/libhdf5_cpp.so;/home/geoffrey/.conda/envs/caffe_27/lib/libhdf5.so;/usr/lib/librt.so;/usr/lib/libpthread.so;/home/geoffrey/.conda/envs/caffe_27/lib/libz.so;/usr/lib/libdl.so;/usr/lib/libm.so;/home/geoffrey/.conda/envs/caffe_27/lib/libhdf5_hl_cpp.so;/home/geoffrey/.conda/envs/caffe_27/lib/libhdf5_hl.so;PUBLIC;/home/geoffrey/.conda/envs/caffe_27/lib/liblmdb.so;PUBLIC;/home/geoffrey/.conda/envs/caffe_27/lib/libleveldb.so;PRIVATE;/home/geoffrey/.conda/envs/caffe_27/lib/libsnappy.so;PUBLIC;/opt/cuda/lib64/libcudart.so;/opt/cuda/lib64/libcurand.so;/opt/cuda/lib64/libcublas.so;PUBLIC;/usr/lib/libcudnn.so;/home/geoffrey/.conda/envs/caffe_27/lib/libnccl.so;PUBLIC;opencv_core;opencv_highgui;opencv_imgproc;opencv_imgcodecs;PUBLIC;/home/geoffrey/.conda/envs/caffe_27/lib/liblapack.so;/home/geoffrey/.conda/envs/caffe_27/lib/libcblas.so;/home/geoffrey/.conda/envs/caffe_27/lib/libblas.so;PRIVATE;/home/geoffrey/.conda/envs/caffe_27/lib/libpython2.7.so;PUBLIC;/home/geoffrey/.conda/envs/caffe_27/lib/libboost_python.so
-- Configuring done
-- Generating done
-- Build files have been written to: /home/geoffrey/caffe/build
(4) 编译
make all -j8
如果出现一下字样, 编译成功.
用pycaffe测试一下:
2. 问题搜集
-
".Doxyfile" cannot be read.`问题
解决方法:
Doxyfile是文档系统, 卸载掉本地的即可. -
未定义的应用
安装了libopencv, 但是缺少头文件, 我conda install opencv解决.
本站文章如无特殊说明，均为本站原创，如若转载，请注明出处：最便捷的caffe编译方法 —- cmake+anaconda虚拟环境 - Python技术站
微信扫一扫 
支付宝扫一扫 