centos7下安装caffe以及pycaffe流程及问题解决

一、环境准备

安装依赖包

python、tesseract依赖：

yum install openssl openssl-devel gcc gcc-c++ autoconf automake libtool libjpeg libpng libtiff zlib libjpeg-devel libpng-devel libtiff-devel zlib-devel python-devel ImageMagick ImageMagick-devel ghostscript python-imaging vim (libyaml)

caffe依赖：

yum install epel-release protobuf-devel leveldb-devel snappy-devel opencv-devel boost-devel hdf5-devel gflags-devel glog-devel lmdb-devel atlas-devel  openblas-devel

源码位置

caffe: https://github.com/BVLC/caffe

下载部署包

• python366：https://www.python.org/downloads/release/python-366/

二、安装Python3

• （1）创建文件夹：

mkdir /usr/local/python3

• （2）解压下载的文件：
  tar xvJf Python-3.6.6.tar.xz
• （3）进入解压后目录，执行编译命令

./configure --prefix=/usr/local/python3
make && make install

• （4）创建软连接命令：

ln -s /usr/local/python3/bin/python3 /usr/bin/python3
ln -s /usr/local/python3/bin/pip3 /usr/bin/pip3

三、安装caffe

• （1）下载额外的rpm包（可能yum源没有），然后安装。

gflags-2.1.1-6.el7.x86_64.rpm        glog-devel-0.3.3-8.el7.x86_64.rpm    leveldb-1.12.0-11.el7.x86_64.rpm        libaec-devel-1.0.4-1.el7.x86_64.rpm  lmdb-libs-0.9.22-2.el7.x86_64.rpm
gflags-devel-2.1.1-6.el7.x86_64.rpm  hdf5-1.8.12-10.el7.x86_64.rpm        leveldb-devel-1.12.0-11.el7.x86_64.rpm  lmdb-0.9.22-2.el7.x86_64.rpm
glog-0.3.3-8.el7.x86_64.rpm          hdf5-devel-1.8.12-10.el7.x86_64.rpm  libaec-1.0.4-1.el7.x86_64.rpm           lmdb-devel-0.9.22-2.el7.x86_64.rpm

• （2）yum安装依赖包。

yum install epel-release protobuf-devel leveldb-devel snappy-devel opencv-devel boost-devel hdf5-devel gflags-devel glog-devel lmdb-devel atlas-devel

-（3）解压caffe源码包，然后执行：

cp Makefile.config.example Makefile.config

-（4）修改文件“Makefile.conf”，内容如下：

CPU_ONLY := 1
USE_OPENCV := 0


WITH_PYTHON_LAYER := 1
 
 
……
 PYTHON_INCLUDE := /usr/local/python3/include/python3.6m \
                 /usr/local/python3/lib/python3.6/site-packages/numpy/core/include \
                 /usr/local/python3/

• （5）开始执行编译：

make all -j8
make test -j16
make runtest

注意：这里“-j8”等参数，表示使用多少个线程进行编译。

• （6） 编译pycaffe。

make pycaffe -j8

然后将编译完成之后的目录“/opt/soft/caffe/caffe-master/python/caffe”拷贝到python的site-packages目录中。

• 异常1: 遇到如下错误：

[root@aidm-1 caffe-master]# vim Makefile.config
[root@aidm-1 caffe-master]# make all -j8
LD -o .build_release/lib/libcaffe.so.1.0.0
/usr/bin/ld: 找不到 -lcblas
/usr/bin/ld: 找不到 -latlas
collect2: 错误：ld 返回 1
make: *** [.build_release/lib/libcaffe.so.1.0.0] 错误 1

则进入 “usr/lib63/atlas”，创建两个软连接：

sudo ln -sv libsatlas.so.3.10 libcblas.so
sudo ln -sv libsatlas.so.3.10 libatlas.so

然后修改“Makefile.config”文件，在“LIBRARY_DIRS”后面追加 “/usr/lib64/atlas”。

• 异常2: 安装完成后，通过python调用出现如下错误：

[root@aidm-1 ~]# python3
Python 3.6.6 (default, Jun 26 2019, 17:33:11)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/python3/lib/python3.6/site-packages/caffe/__init__.py", line 1, in <module>
    from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer
  File "/usr/local/python3/lib/python3.6/site-packages/caffe/pycaffe.py", line 13, in <module>
    from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \
ImportError: /usr/local/python3/lib/python3.6/site-packages/caffe/_caffe.so: undefined symbol: _ZN5boost6python6detail11init_moduleER11PyModuleDefPFvvE

要在“Makefile.config”文件中去掉这行注释：

WITH_PYTHON_LAYER := 1

然后，在/usr/lib64中添加如下两个文件：

libpython3.4m.so.1.0
libboost_python34.so.1.69.0

然后，重新编译。

• 异常3: python调用出现如下错误：

Failed to include caffe_pb2, things might go wrong!
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/abc/workplace/caffe/python/caffe/__init__.py", line 4, in <module>
    from .proto.caffe_pb2 import TRAIN, TEST
  File "/home/abc/workplace/caffe/python/caffe/proto/caffe_pb2.py", line 17, in <module>
    serialized_pb='\n\x0b\x63\x61\x66\x66\x65.proto\x12\x05\x63\x61\x66\x66\x65\"\x1c\n\tBlobShape\x12\x0f\n\x03\x64i...'
  File "/home/abc/anaconda3/lib/python3.6/site-packages/google/protobuf/descriptor.py", line 824, in __new__
    return _message.default_pool.AddSerializedFile(serialized_pb)
TypeError: expected bytes, str found

原因是之前编译的python包“caffe”中的proto/caffe_pb2.py文件有问题，找一个正确的替换下即可。

• 有用的linux命令

ls -l|grep "7月 2" | awk -F ' ' '{print $9;}' | xargs -i cp {} /home/aidm-algorithm/libs/ext-lib-all

问题

安装Python完成之后，执行pip3 list 命令之后报如下错误：

[root@aidm-2 ~]# pip3 list
Package    Version
---------- -------
pip        10.0.1
setuptools 39.0.1
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)) - skipping

• 原因：新版本pip默认使用https，而Python里面没有找到ssl模块（由于编译python的时候没有指定）。
• 解决方式1: 配置一个http的pip源，然后将这个源的域名设置为受信任的，参考操作如下：

mkdir ~/.pip
cd ~/.pip 
touch pip.conf

然后添加如下内容：

[global] 
index-url = http://pypi.douban.com/simple 
[install]
trusted-host = pypi.douban.com

• 解决方式2: 首先检查系统中是否包含openssl和opeenssl-devel两个包，如果没有，则首先进行安装。然后重新编译python：

./configure --prefix=/usr/local/python3  --with-ssl
make && make install