Overview
Modern websites using SPA (Single Page Applications) or Ajax often do not show all elements immediately after a page opens. Elements may appear with a slight delay. To operate these elements without errors in Selenium, “wait processing” is essential to temporarily pause the program until the element appears. This article explains two methods: “Implicit Wait” and “Explicit Wait.”
Specifications (Input/Output)
- Implicit Wait (implicitly_wait):
- Input: Maximum number of seconds to wait.
- Effect: Applied to the entire life of the driver. It polls for elements for the specified duration if they are not immediately found.
- Explicit Wait (WebDriverWait):
- Input: Driver, maximum seconds, and an Expected Condition (EC).
- Effect: Only waits at a specific point until a condition (such as an element becoming visible or clickable) is met.
Basic Usage
“Implicit Wait” is the easiest method. It tells the driver to wait for a maximum number of seconds if an element is not found.
from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
# Set Implicit Wait (Max 5 seconds)
# From now on, find_element will search for up to 5 seconds if the element is missing
driver.implicitly_wait(5)
try:
driver.get("https://example.com")
# Even if it doesn't appear immediately, it can be retrieved if it shows up within 5 seconds
element = driver.find_element(By.TAG_NAME, "h1")
print(element.text)
finally:
driver.quit()
Full Code
This implementation uses “Explicit Wait (WebDriverWait),” which is more flexible and recommended for professional use. It waits specifically for the “visibility” of an element, not just its existence in the DOM.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
def wait_for_element_demo():
"""
Demo function using WebDriverWait to wait until a specific element is visible.
"""
driver = webdriver.Chrome()
# Define wait time (Max 10 seconds)
wait = WebDriverWait(driver, 10)
try:
# Assuming we open a site with delayed loading
driver.get("https://morinokabu.com")
print("Waiting for the element to appear...")
# Execute Explicit Wait
# until(condition): Waits until the condition is True or a WebElement is returned
# EC.visibility_of_element_located: Checks that the element is in the DOM and visible (size > 0)
target_element = wait.until(
EC.visibility_of_element_located((By.TAG_NAME, "h1"))
)
# Once the wait is over, the element is visible
print(f"Element retrieved: {target_element.text}")
except TimeoutException:
print("The element was not displayed within the specified time (Timeout).")
except Exception as e:
print(f"Unexpected error: {e}")
finally:
driver.quit()
if __name__ == "__main__":
wait_for_element_demo()
Customization Points
Differences Between the Two Wait Methods
| Method | Syntax | Features / Purpose |
| implicitly_wait | driver.implicitly_wait(seconds) | Global setting. Applies to all find_element calls. Good for simple scripts. |
| WebDriverWait | WebDriverWait(driver, seconds).until(...) | One-time execution. Can wait for detailed conditions like “visible” or “clickable.” More stable and recommended for work. |
Frequently Used Expected Conditions (EC)
Use these by importing from selenium.webdriver.support import expected_conditions as EC.
- EC.presence_of_element_located((By.ID, “id”))Checks if the element exists in the HTML (it does not have to be visible).
- EC.visibility_of_element_located((By.ID, “id”))Checks if the element exists and is visible on the screen (height and width > 0).
- EC.element_to_be_clickable((By.ID, “id”))Checks if the element is visible and enabled so that you can click it.
Important Notes
Do Not Mix Methods
Do not use implicitly_wait and WebDriverWait together. This can cause unpredictable wait times or unstable behavior. Use one or the other (generally WebDriverWait).
Double Parentheses for Tuples
Arguments passed to EC methods must be a single tuple: (By.ID, “name”).
Note that you must use double parentheses:
EC.presence_of_element_located((By.ID, “name”)) instead of EC.presence_of_element_located(By.ID, “name”).
Timeout Exceptions
If a condition is not met within the specified time, a TimeoutException occurs. Always wrap your code in a try-except block to handle errors gracefully, such as by logging or retrying.
Advanced Usage
This is a helper function for the safest operation pattern: “Wait until a button is clickable, then click it.”
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
def safe_click(driver, locator, timeout=10):
"""
Helper function that waits for an element to be clickable before clicking.
"""
try:
element = WebDriverWait(driver, timeout).until(
EC.element_to_be_clickable(locator)
)
element.click()
print("Successfully clicked")
except Exception:
print("Could not click the element")
# Usage Example
if __name__ == "__main__":
driver = webdriver.Chrome()
driver.get("https://example.com")
# Safely click the button with ID="submit-btn"
safe_click(driver, (By.ID, "submit-btn"))
driver.quit()
Summary
Instability (flakiness) in scraping or automated testing is often caused by poor wait management. While implicitly_wait is easy to use, you should master WebDriverWait and ExpectedConditions for reliable performance. Always clearly define “what state you are waiting for.”
