Parallel

parallel allows the user to execute shell scripts or commands in parallel.

• Installation: brew install parallel
• parallel --help

Examples

aws ec2 describe-regions --query "Regions[].{Name:RegionName}" --output text | parallel "echo {}; aws ec2 describe-instances --region {}"

your_command | parallel "echo {}; your_command2 {}"

ls -d */ | parallel git -C {} fetch
ls -d */ | parallel git -C {} pull

parallel -j 12 --eta rsync -a {} /mnt/efs/myexample ::: mydest/*

Activities

Basic

1. Execute a command in parallel by ssh in MACHINE1 and MACHINE2 using parallel
2. Read stackexchange questions about gnu-parallel: https://unix.stackexchange.com/questions/tagged/gnu-parallel?tab=votes&pagesize=50
3. Install parallel in macOS: brew install parallel can conflict with brew install moreutils

Intermediate

1. Review different parallel options such as: --keep-order or -k which keep order in the output. Do not affect order of execution.
2. Update all your git repositories in a directory in macOS using parallel: ls | parallel git -C {} fetch

Related terms

• Parallels
• for
• pssh
• parallel --citation
• yarn workspaces foreach --paralell
• PostgreSQL parallel query
• Android parallel apps
• Parallel computing
• ansible -B, --background
• parallel --sshlogin
• pigz, gipz parallel implementation

See also

• parallel, for, xargs, parallel --citation, sshlogin
• xargs, parallel, paste, map
• moreutils
• stress, stress-ng, parallel, ab
• CPU, GPU, NPU, TPU, DPU, Groq, Proliant, thread (Pthreads), processor, CPU socket, core, ARM , CPU Virtualization, Intel, AMD, nm, lscpu, AVX-512, Passthrough, CPU intensive, Graviton processor, Branch predictor, vCPU, SSE, Power, CPU cache

Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy.

Source: https://en.wikiversity.org/wiki/Linux/Basic_commands/parallel