一、环境准备

安装依赖包

python、tesseract依赖:

yum install openssl openssl-devel gcc gcc-c++ autoconf automake libtool libjpeg libpng libtiff zlib libjpeg-devel libpng-devel libtiff-devel zlib-devel python-devel ImageMagick ImageMagick-devel ghostscript python-imaging vim (libyaml)

caffe依赖:

yum install epel-release protobuf-devel leveldb-devel snappy-devel opencv-devel boost-devel hdf5-devel gflags-devel glog-devel lmdb-devel atlas-devel  openblas-devel

源码位置

caffe: https://github.com/BVLC/caffe

下载部署包

二、安装Python3

  • (1)创建文件夹:
mkdir /usr/local/python3
  • (2)解压下载的文件:
    tar xvJf Python-3.6.6.tar.xz
  • (3)进入解压后目录,执行编译命令
./configure --prefix=/usr/local/python3
make && make install
  • (4)创建软连接命令:
ln -s /usr/local/python3/bin/python3 /usr/bin/python3
ln -s /usr/local/python3/bin/pip3 /usr/bin/pip3

三、安装caffe

  • (1)下载额外的rpm包(可能yum源没有),然后安装。
gflags-2.1.1-6.el7.x86_64.rpm        glog-devel-0.3.3-8.el7.x86_64.rpm    leveldb-1.12.0-11.el7.x86_64.rpm        libaec-devel-1.0.4-1.el7.x86_64.rpm  lmdb-libs-0.9.22-2.el7.x86_64.rpm
gflags-devel-2.1.1-6.el7.x86_64.rpm  hdf5-1.8.12-10.el7.x86_64.rpm        leveldb-devel-1.12.0-11.el7.x86_64.rpm  lmdb-0.9.22-2.el7.x86_64.rpm
glog-0.3.3-8.el7.x86_64.rpm          hdf5-devel-1.8.12-10.el7.x86_64.rpm  libaec-1.0.4-1.el7.x86_64.rpm           lmdb-devel-0.9.22-2.el7.x86_64.rpm
  • (2)yum安装依赖包。
yum install epel-release protobuf-devel leveldb-devel snappy-devel opencv-devel boost-devel hdf5-devel gflags-devel glog-devel lmdb-devel atlas-devel

-(3)解压caffe源码包,然后执行:

cp Makefile.config.example Makefile.config

-(4)修改文件“Makefile.conf”,内容如下:

CPU_ONLY := 1
USE_OPENCV := 0


WITH_PYTHON_LAYER := 1
 
 
……
 PYTHON_INCLUDE := /usr/local/python3/include/python3.6m \
                 /usr/local/python3/lib/python3.6/site-packages/numpy/core/include \
                 /usr/local/python3/

  • (5)开始执行编译:
make all -j8
make test -j16
make runtest

注意:这里“-j8”等参数,表示使用多少个线程进行编译。

  • (6) 编译pycaffe。
make pycaffe -j8

然后将编译完成之后的目录“/opt/soft/caffe/caffe-master/python/caffe”拷贝到python的site-packages目录中。

  • 异常1: 遇到如下错误:
[root@aidm-1 caffe-master]# vim Makefile.config
[root@aidm-1 caffe-master]# make all -j8
LD -o .build_release/lib/libcaffe.so.1.0.0
/usr/bin/ld: 找不到 -lcblas
/usr/bin/ld: 找不到 -latlas
collect2: 错误:ld 返回 1
make: *** [.build_release/lib/libcaffe.so.1.0.0] 错误 1

则进入 “usr/lib63/atlas”,创建两个软连接:

sudo ln -sv libsatlas.so.3.10 libcblas.so
sudo ln -sv libsatlas.so.3.10 libatlas.so

然后修改“Makefile.config”文件,在“LIBRARY_DIRS”后面追加 “/usr/lib64/atlas”。

  • 异常2: 安装完成后,通过python调用出现如下错误:
[root@aidm-1 ~]# python3
Python 3.6.6 (default, Jun 26 2019, 17:33:11)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/python3/lib/python3.6/site-packages/caffe/__init__.py", line 1, in <module>
    from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer
  File "/usr/local/python3/lib/python3.6/site-packages/caffe/pycaffe.py", line 13, in <module>
    from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \
ImportError: /usr/local/python3/lib/python3.6/site-packages/caffe/_caffe.so: undefined symbol: _ZN5boost6python6detail11init_moduleER11PyModuleDefPFvvE

要在“Makefile.config”文件中去掉这行注释:

WITH_PYTHON_LAYER := 1

然后,在/usr/lib64中添加如下两个文件:

libpython3.4m.so.1.0
libboost_python34.so.1.69.0

然后,重新编译。

  • 异常3: python调用出现如下错误:
Failed to include caffe_pb2, things might go wrong!
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/abc/workplace/caffe/python/caffe/__init__.py", line 4, in <module>
    from .proto.caffe_pb2 import TRAIN, TEST
  File "/home/abc/workplace/caffe/python/caffe/proto/caffe_pb2.py", line 17, in <module>
    serialized_pb='\n\x0b\x63\x61\x66\x66\x65.proto\x12\x05\x63\x61\x66\x66\x65\"\x1c\n\tBlobShape\x12\x0f\n\x03\x64i...'
  File "/home/abc/anaconda3/lib/python3.6/site-packages/google/protobuf/descriptor.py", line 824, in __new__
    return _message.default_pool.AddSerializedFile(serialized_pb)
TypeError: expected bytes, str found

原因是之前编译的python包“caffe”中的proto/caffe_pb2.py文件有问题,找一个正确的替换下即可。

  • 有用的linux命令

ls -l|grep "7月 2" | awk -F ' ' '{print $9;}' | xargs -i cp {} /home/aidm-algorithm/libs/ext-lib-all

问题

安装Python完成之后,执行pip3 list 命令之后报如下错误:

[root@aidm-2 ~]# pip3 list
Package    Version
---------- -------
pip        10.0.1
setuptools 39.0.1
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)) - skipping

  • 原因:新版本pip默认使用https,而Python里面没有找到ssl模块(由于编译python的时候没有指定)。
  • 解决方式1: 配置一个http的pip源,然后将这个源的域名设置为受信任的,参考操作如下:
mkdir ~/.pip
cd ~/.pip 
touch pip.conf

然后添加如下内容:

[global] 
index-url = http://pypi.douban.com/simple 
[install]
trusted-host = pypi.douban.com

  • 解决方式2: 首先检查系统中是否包含openssl和opeenssl-devel两个包,如果没有,则首先进行安装。然后重新编译python:
./configure --prefix=/usr/local/python3  --with-ssl
make && make install