概述

在工作当中执行某个脚本时常常因为循规蹈矩的进度导致缓慢，需要通过多线程来进行解决速度慢的问题。

生成测试文件，模拟多个任务

[root@VM_0_9_centos ~]# for num in {1..10};do echo $num >> test.txt;done

单进程

[root@VM_0_9_centos ~]# cat test.sh
#!/bin/bash
`File_Name=${1:?we need a parameter!}
if [ ! -f "${File_Name}" ];then
echo "Can't find the file ${File_Name} !"
exit 1
fi
while read ;do
{
echo "$REPLY"
sleep 1
}
done < ${File_Name}
echo done`

运行:

[root@VM_0_9_centos ~]# time ./test.sh test.txt
1
2
3
4
5
6
7
8
9
10
done
`real    0m10.014s
user    0m0.006s
sys     0m0.008s`

多进程

&实现多进程，wait实现父进程等待子进程完成后退出

[root@VM_0_9_centos ~]# cat test.sh
#!/bin/bash
`File_Name=${1:?we need a parameter!}
if [ ! -f "${File_Name}" ];then
echo "Can't find the file ${File_Name} !"
exit 1
fi
while read ;do
{
echo "$REPLY"
sleep 1
} &
done < ${File_Name}
wait
echo done`

运行，数字全部一起输出，耗时1s：

[root@VM_0_9_centos ~]# time ./test.sh test.txt
1
2
7
3
8
4
9
5
10
6
done
`real    0m1.010s
user    0m0.012s
sys     0m0.006s`

通过管道控制进程数

[root@VM_0_9_centos ~]# cat test.sh
#!/bin/bash

File_Name=${1:?we need a parameter!}
if \[ ! -f "${File_Name}" \];then
echo "Can't find the file ${File_Name} !"
exit 1
fi


#线程数
thread_num=4


create pipe
===========



\[ ! -p tmp \] \&\& mkfifo tmp
exec 9\<\>tmp


fill in the pipe
================



for ((i=0;i\<thread_num;i++)); do
echo 1\>\&9
done

`while read ;do
# remove one line per thread
(read -u 9) # In order to avoid the "read" command in while loop, we use the parentheses here
{
echo $REPLY
sleep 1
# thread run finish, put line back
echo 1>&9
} &
done < ${File_Name}
wait
echo done`

运行如下，数字四个一组输出，共三组，耗时3s：

[root@VM_0_9_centos ~]# time ./test.sh test.txt
1
2
3
4
5
6
7
8
9
10
done
`real    0m3.017s
user    0m0.012s
sys     0m0.011s`

另辟蹊径，通过 xargs 制进程数

[root@VM_0_9_centos ~]# cat test.sh
#!/bin/bash

whereami=`cd $(dirname $0); pwd`
file_name=${1:?we need a parameter!}
if \[ ! -f "${file_name}" \];then
echo "Can't find the file ${file_name} !"
exit 1
fi


#线程数
thread_num=4


cat ${file_name} \|xargs -n 1 -I {} -P ${thread_num} sh -c "$whereami/print.sh {}"
echo done


\[root@VM_0_9_centos \~\]# cat print.sh
#!/bin/bash

`echo $*
sleep 1`

运行如下，数字四个一组输出，共三组，耗时3s：

[root@VM_0_9_centos ~]# time ./test.sh test.txt
1
2
3
4
5
6
7
8
9
10
done
`real    0m3.022s
user    0m0.025s
sys     0m0.023s`