pyinstall 打包 python代码为可执行文件（pytorch）

利用pyinstaller(4.2)打包pytorch，开始使用的python版本为3.7.4，在Ubuntu18.04上能打包成功，但在windows10上一直报错numpy.core.multiarray failed to import，尝试了很多方法，最终在import torch之前添加import numpy后打包成功。

一、代码

testTorch.py

#import numpy
import torch

def test():
    #print("Numpy: ", numpy.array(3))
    print("Torch: ", torch.tensor(3))
    print("CUDA: ", torch.cuda.is_available())
    
if __name__ == "__main__":
    test()

二、环境

python: 3.7.4

torch: 1.5.0

numpy: 1.20.1

pyinstaller: 4.2

三、打包

Pyinstaller教程链接：https://pyinstaller.readthedocs.io/en/stable/installation.html

pyinstaller -F -c testTorch.py    // -F: 生成一个exe文件， -c: 运行时打印后台信息

如果更改配置文件testTorch.spec，打包的时候运行
$ pyinstaller -F -c testTorch.spec
再次进行打包

3.1 Window10

Error info

ImportError: numpy.core.multiarray failed to import

Traceback (most recent call last):
  File "testTorch.py", line 1, in <module>
    import torch
  File "PyInstaller\loader\pyimod03_importers.py", line 531, in exec_module
  File "torch\__init__.py", line 136, in <module>
ImportError: numpy.core.multiarray failed to import
[10792] Failed to execute script testTorch

Solutions

s1
更改终端模式
失败
s2
修改testTorch.spec文件
在hiddenimports=[]中添加'numpy', 'numpy.core.multiarray'
失败
s3
更改numpy版本
 https://www.datasciencelearner.com/importerror-numpy-core-multiarray-failed-to-import/
失败
s4
在运行的py文件加入以下命令
```
import numpy.core._dtype_ctypes
```
失败
s5
在import torch之前加入import numpy，重新编译

成功

3.2 Ubuntu18.04

Error info

OSError: could not get source code

Traceback (most recent call last):
  File "torch/_utils_internal.py", line 46, in get_source_lines_and_file
  File "inspect.py", line 955, in getsourcelines
  File "inspect.py", line 786, in findsource
OSError: could not get source code

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "testTorch.py", line 1, in <module>
    import torch
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "torch/__init__.py", line 367, in <module>
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "torch/distributions/__init__.py", line 112, in <module>
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "PyInstaller/loader/pyimod03_importers.py", line 531, in exec_module
  File "torch/distributions/von_mises.py", line 54, in <module>
  File "torch/jit/__init__.py", line 1287, in script
  File "torch/jit/frontend.py", line 164, in get_jit_def
  File "torch/_utils_internal.py", line 53, in get_source_lines_and_file
OSError: Can't get source for <function _rejection_sample at 0x7fcf5fc8a170>. TorchScript requires source access in order to carry out compilation, make sure original .py files are available. Original error: could not get source code
[27633] Failed to execute script testTorch

Solutions

修改testTorch.spec文件，如下：

block_cipher = None
excluded_modules = ['torch.distributions'] # 加入这一行

a = Analysis(['xxx.py'],
             pathex=['path'],
             binaries=[],
             datas=[],
             hiddenimports=[],
             hookspath=[],
             runtime_hooks=[],
             excludes=excluded_modules, # 把=[]改成=excluded_modules
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher,
             noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)

再重新打包

$ pyinstaller -F testTorch.spec

成功

四、其他问题

Error: No module named ‘A’

解决办法
在*.spec文件中的hiddenimports中添加相应的A模块，重新编译；
或者开始打包的时候在命令行后面添加--hidden-import **A**。

WARNING: lib not found: torch_python.dll

解决办法
在打包命令行中加入--paths dll_dir,其中dll_dir表示dll所在路径。

WARNING: file already exists but should not: C:\Users\ADMINI~1\AppData\Local\Temp_MEI201482\torch_C.cp36-win_amd64.pyd

解决办法
在*.spec文件中的a = Analysis(...）下方加入以下命令，然后重新打包pyinstaller -F *.spec.

for d in a.datas:
  if '_C.cp36-win_amd64.pyd' in d[0]:
    a.datas.remove(d)
    break

路径问题

方案1 修改为相对路径
在python程序中可能会依赖其他文件，例如res/test.txt等，可以在代码中修改test.txt的路径为相对路径

print("argv[0]: ", os.path.dirname(os.path.realpath(sys.argv[0])))
test_path = os.path.dirname(os.path.realpath(sys.argv[0])) + "res/test.txt"

打包完成后将res/test.txt复制到exe所在路径即可。

方案2 将资源文件一起打包
参考链接： https://www.cnblogs.com/darcymei/p/9397173.html
- 首先修改python代码路径

#coding:utf-8
import sys
import os

#生成资源文件目录访问路径
def resource_path(relative_path):
    if getattr(sys, 'frozen', False): #是否Bundle Resource
        base_path = sys._MEIPASS
    else:
        base_path = os.path.abspath(".")
    return os.path.join(base_path, relative_path)

#访问res文件夹下test.txt的内容
filename = resource_path(os.path.join("res","test.txt"))
print(filename)
with open(filename) as f:
    lines = f.readlines()
    print(lines)
    f.close()