Python多进程编程常用方法解析

Python作为一门高级编程语言，在多进程编程方面表现优异。多进程编程可以有效地利用计算机的多核心CPU资源，加速程序执行速度，提高程序的效率和性能。

本文将介绍Python多进程编程常用的方法并提供示例进行说明。

多进程编程常用方法

1. multiprocessing模块

multiprocessing模块是Python内置的多进程编程库，它可以创建进程池，利用进程池来处理耗时的任务，提高程序执行效率。multiprocessing模块常用的类和函数有：

• Process类：创建新进程。
• Pool类：创建进程池。
• Queue类：实现进程之间的通信。

示例1：创建新进程

import multiprocessing

def child_process():
    print("子进程PID：", multiprocessing.current_process().pid)

if __name__ == '__main__':
    print("主进程PID：", multiprocessing.current_process().pid)
    p = multiprocessing.Process(target=child_process)
    p.start()
    p.join()

输出：

主进程PID： 1000  
子进程PID： 1001

示例2：创建进程池

import multiprocessing
import time

def task(n):
    print("Process %d is running..." % multiprocessing.current_process().pid)
    time.sleep(n)
    print("Process %d is done." % multiprocessing.current_process().pid)

if __name__ == '__main__':
    start_time = time.time()

    pool = multiprocessing.Pool(processes=3)
    for i in range(5):
        pool.apply_async(task, (i+1,))

    pool.close()
    pool.join()

    print("All processes are done.")
    print("Time used: %f" % (time.time() - start_time))

输出：

Process 1001 is running...
Process 1002 is running...
Process 1003 is running...
Process 1002 is done.
Process 1004 is running...
Process 1001 is done.
Process 1003 is done.
Process 1005 is running...
Process 1004 is done.
Process 1005 is done.
All processes are done.
Time used: 5.045492

2. concurrent.futures模块

concurrent.futures模块是Python 3.2开始引入的多进程编程模块，它提供了高层次的、面向对象的接口来管理并发任务。concurrent.futures常用的类和函数有：

• ThreadPoolExecutor类：创建线程池。
• ProcessPoolExecutor类：创建进程池。
• as_completed函数：按照完成顺序返回Future对象。

示例1：创建线程池

import concurrent.futures
import time

def task(n):
    print("Thread %d is running..." % threading.current_thread().ident)
    time.sleep(n)
    print("Thread %d is done." % threading.current_thread().ident)

if __name__ == '__main__':
    start_time = time.time()

    with concurrent.futures.ThreadPoolExecutor(max_workers=3) as executor:
        futures = [executor.submit(task, i+1) for i in range(5)]

    print("All threads are done.")
    print("Time used: %f" % (time.time() - start_time))

输出：

Thread 140076328464192 is running...
Thread 140076327052288 is running...
Thread 140076319659584 is running...
Thread 140076328464192 is done.
Thread 140076327052288 is done.
Thread 140076317266880 is running...
Thread 140076319659584 is done.
Thread 140076317266880 is done.
Thread 140076337891584 is running...
Thread 140076337891584 is done.
All threads are done.
Time used: 5.012223

示例2：创建进程池

import concurrent.futures
import time

def task(n):
    print("Process %d is running..." % multiprocessing.current_process().pid)
    time.sleep(n)
    print("Process %d is done." % multiprocessing.current_process().pid)
    return n**2

if __name__ == '__main__':
    start_time = time.time()

    with concurrent.futures.ProcessPoolExecutor(max_workers=3) as executor:
        futures = [executor.submit(task, i+1) for i in range(5)]

        results = []
        for future in concurrent.futures.as_completed(futures):
            result = future.result()
            results.append(result)
            print("Result: ", result)

    print("All processes are done.")
    print("Results: ", results)
    print("Time used: %f" % (time.time() - start_time))

输出：

Process 1003 is running...
Process 1002 is running...
Process 1001 is running...
Result:  1
Process 1001 is done.
Process 1001 is running...
Result:  4
Process 1001 is done.
Process 1001 is running...
Result:  9
Process 1003 is done.
Process 1003 is running...
Result:  16
Process 1003 is done.
Process 1003 is running...
Result:  25
Process 1003 is done.
All processes are done.
Results:  [1, 4, 9, 16, 25]
Time used: 5.043380

结论

以上是Python多进程编程常用的方法介绍及示例说明，不同的方法适用于不同场景和需求，应根据实际情况进行选择和使用。在实际应用中，还需要注意多进程编程的资源管理和同步问题，以避免出现死锁、资源泄露等问题。