When you want to extract only specific data that matches a condition from a list (array) in Python, the built-in filter() function is very convenient.
While you can use a combination of for loops and if statements, using filter() makes the code more concise and clearly states your intent: that you are filtering data.
The Problem to Solve
This method is used to create a new list containing only the elements that meet specific criteria (True/False) from a large dataset. Common use cases include extracting passing scores, finding in-stock products, or isolating specific error logs.
Implementation Example: Extracting Valid Sensor Data
In this scenario, we extract only valid data (values 0 or greater) from a list of sensor readings, excluding abnormal noise values (negative numbers).
Source Code
# Raw sensor data (negative values are considered errors)
sensor_values = [105, -1, 230, 45, -99, 12, 0, 310]
# Function to define filtering conditions
# Returns True if 0 or more (keep), False if negative (exclude)
def is_valid_data(val):
return val >= 0
# Execute filter function
# 1st argument: The function performing the check
# 2nd argument: The target list
# Return value is an iterator, so convert to list() to finalize
valid_values = list(filter(is_valid_data, sensor_values))
# Output results
print(f"Original Data: {sensor_values}")
print(f"Filtered Data: {valid_values}")
Execution Result
Original Data: [105, -1, 230, 45, -99, 12, 0, 310]
Filtered Data: [105, 230, 45, 12, 0, 310]
Explanation
How filter() Works
filter(function, iterable) passes each element of the iterable (such as a list) to the function. It creates an iterator that keeps only the elements where the function returns True.
- 1st Argument: Specify a function that returns
TrueorFalse. (If you specifyNone, it extracts only elements that evaluate to true, filtering out 0 or empty strings). - 2nd Argument: Specify the list or tuple to be filtered.
Simplification using Lambda Expressions
If the logic for the condition is simple, it is common to use a lambda (anonymous function) to write it in a single line, rather than defining a full function with def.
The code above can be rewritten using lambda as follows:
# Omit function definition and write the condition directly with lambda
# Extract only elements where val >= 0
valid_values = list(filter(lambda val: val >= 0, sensor_values))
This concise style using lambda expressions is frequently used in production code.
