[Linux] Execute Commands in Bulk with Standard Input Using the xargs Command

目次

Overview

The xargs command takes a list of data from standard input (such as a pipeline) and converts it into “arguments” for a specific command.

It is a key tool for pipeline processing and automation. For example, you can use it to delete a large number of files found by the find command or create multiple directories from a text list. It significantly reduces the need for manual repetition.


Specifications (Arguments and Options)

Syntax

xargs [options] [command [arguments...]]

Main Arguments and Options

OptionDescription
-0 / --nullTreats input as separated by null characters (\0). This prevents errors when filenames contain spaces.
-a <file>Reads items from a specific file instead of standard input.
-d <delimiter>Specifies a custom character to separate inputs (default is space or newline).
-I <string>Defines a placeholder (e.g., {}) to insert arguments into a specific part of a command.
-n <number>Sets the maximum number of arguments to pass per command execution.
-pAsks the user for confirmation before running each command (interactive).
-P <number>Runs commands in parallel using the specified number of processes for faster speed.
-tPrints the command to standard error before executing it (trace).

Basic Usage

The command takes strings passed through a pipe and gives them to the next command as arguments.

Command

# Pass three filenames to ls -l as arguments
echo "file1.txt file2.txt file3.txt" | xargs ls -l

Execution Result

xargs expands the input, so the system actually runs ls -l file1.txt file2.txt file3.txt.

-rw-r--r-- 1 user user 0 Jan 20 10:00 file1.txt
-rw-r--r-- 1 user user 0 Jan 20 10:00 file2.txt
-rw-r--r-- 1 user user 0 Jan 20 10:00 file3.txt

Practical Commands

Safely Delete Large Numbers of Files (Handling Spaces)

If a filename contains a space, standard pipes might split the name and cause errors. Combining find -print0 with xargs -0 ensures that filenames are handled safely using null delimiters.

# Find files ending in .tmp and delete them safely
find . -name "*.tmp" -print0 | xargs -0 rm -vf

Plaintext

removed './test file.tmp'
removed './data.tmp'

Inserting Arguments at a Specific Position (Copy/Move)

Normally, xargs adds arguments to the end of a command. Use the -I option to place the filename anywhere in the command.

# Copy all .jpg files to the backup/ directory
# "{}" is replaced by each input filename
ls *.jpg | xargs -I {} cp {} ./backup/

Adding Confirmation Before Execution (Preventing Mistakes)

When using dangerous commands like rm, use the -p option. It shows the command line and waits for you to press y before executing.

# Confirm before deleting log files
ls *.log | xargs -p rm
rm app.log error.log ?...

(The files are deleted only if you type y and press Enter.)

Loading a List from a File for Parallel Downloads

Use -a to read a list and -P to run processes in parallel. This is effective for downloading images quickly from a URL list.

# Read url_list.txt and run wget in 4 parallel processes
xargs -a url_list.txt -P 4 -n 1 wget -q

Processing with Custom Delimiters (CSV, etc.)

The -d option allows you to use characters other than spaces (like commas) as separators.

# Treat a comma-separated string as a list and create directories
echo -n "dir_A,dir_B,dir_C" | xargs -d "," mkdir -v
mkdir: created directory 'dir_A'
mkdir: created directory 'dir_B'
mkdir: created directory 'dir_C'

Customization Points

  • Placeholder Names: While -I {} is the standard, you can use clearer names like -I FILENAME.
  • Limiting Argument Counts: xargs tries to pass as many arguments as possible at once. If a command has an argument limit, use -n 1 to run the command once for every single input item.
  • Parallelism: Using -P 0 will launch as many processes as possible. Be careful, as this can put a high load on your CPU.

Important Notes

  • Filenames with Spaces: Commands like ls | xargs rm will fail if filenames contain spaces because xargs will see one file as two. Always use find ... -print0 | xargs -0 ... for file operations.
  • Command Length Limits: Systems have a maximum command-line length (ARG_MAX). xargs handles this by automatically splitting the execution into multiple commands if the list is too long.
  • Execution Without Confirmation: Unless you use -p, commands run immediately. When using rm or mv, it is a good idea to test first using xargs echo to see exactly what commands will be generated.

Applications

Bulk Deletion of Unused Docker Resources

This is a common pattern for cleaning up unused containers or images.

# Get IDs of all stopped Docker containers and delete them at once
docker ps -aq | xargs -r docker rm

# Delete all old images tagged as <none>
docker images -f "dangling=true" -q | xargs -r docker rmi

Note: The -r (no-run-if-empty) option is a GNU extension that prevents the command from running if the input is empty.


Summary

The xargs command is much more than a simple repetition tool; it expands the possibilities of Linux pipelines. Using it with find for file operations or using -P for parallel processing dramatically improves the efficiency of server management and data tasks. By mastering null delimiters (-0) and placeholders (-I), you can execute complex bulk operations safely and quickly.

よかったらシェアしてね!
  • URLをコピーしました!
  • URLをコピーしました!

この記事を書いた人

私が勉強したこと、実践したこと、してることを書いているブログです。
主に資産運用について書いていたのですが、
最近はプログラミングに興味があるので、今はそればっかりです。

目次