[Python] Browser Automation with Selenium: Environment Setup and Basic Mechanics

Overview

This is a basic guide and introduction to “Selenium,” a standard library for automating web browsers in Python. It is widely used for automating web application testing and data collection (web scraping). With the introduction of “Selenium Manager” in version 4.6 and later, the management of WebDrivers is now automated, making the environment setup very simple.

Specifications

• Input: Target URL.
• Output: Launch of the specified browser and page display.
• Prerequisites:
  • A browser such as Google Chrome must be installed.
  • The selenium library must be installed (pip install selenium).

Basic Usage

Install the library and run this basic code to launch Chrome and open the Google homepage. You no longer need to download ChromeDriver separately or set up system paths.

pip install selenium

from selenium import webdriver
import time

# Launch the browser (Driver is automatically retrieved)
driver = webdriver.Chrome()

# Access Google
driver.get("https://www.google.com")

# Wait for 5 seconds to confirm
time.sleep(5)

# Close the browser
driver.quit()

Full Code Example

The following is a complete implementation that launches a browser, retrieves the page title, prints it to the console, and then shuts down safely. It also includes commented-out lines for switching between major browsers (Chrome, Firefox, and Edge).

from selenium import webdriver
from selenium.common.exceptions import WebDriverException
import time

def start_browser_automation():
    """
    Demo function to launch a browser and access a website using Selenium.
    WebDriver installation is handled automatically by Selenium Manager.
    """
    driver = None
    try:
        print("Launching the browser...")
        
        # Select the class according to the browser you want to use
        # For Chrome
        driver = webdriver.Chrome()
        
        # For Firefox
        # driver = webdriver.Firefox()
        
        # For Edge
        # driver = webdriver.Edge()
        
        # For Safari (macOS only, requires additional setup)
        # driver = webdriver.Safari()

        print("Accessing the website...")
        target_url = "https://www.python.org"
        driver.get(target_url)

        # Retrieve page information
        title = driver.title
        current_url = driver.current_url
        
        print("-" * 30)
        print(f"Page Title: {title}")
        print(f"Current URL: {current_url}")
        print("-" * 30)

        # Wait briefly for confirmation
        time.sleep(3)

    except WebDriverException as e:
        print(f"An error occurred during browser operation: {e}")
    
    except Exception as e:
        print(f"Unexpected error: {e}")

    finally:
        # Always close the browser after processing
        if driver:
            print("Closing the browser.")
            driver.quit()

if __name__ == "__main__":
    start_browser_automation()

Customization Points

There are three main elements that make up Selenium. Here are the points for changing settings.

• Python Client Library: This is the part installed via pip install selenium. You give commands through Python code.
• The Browser: Chrome, Firefox, Edge, Safari, etc. This is the application that actually displays the screen. It must be installed on your PC.
• WebDriver: This is the software that acts as a bridge between Python and the browser.
  • Selenium Manager (Selenium 4.6+): This built-in feature detects your installed browser version and automatically downloads and configures the appropriate WebDriver (like ChromeDriver or GeckoDriver).

Driver Classes for Each Browser

Browser	Driver Class	Notes
Google Chrome	webdriver.Chrome()	Most common. Highly stable.
Mozilla Firefox	webdriver.Firefox()	Uses GeckoDriver.
Microsoft Edge	webdriver.Edge()	Based on Chrome, highly compatible.
Apple Safari	webdriver.Safari()	macOS only. Requires “Allow Remote Automation” in the Develop menu.

Important Notes

• Version Mismatch: Before Selenium Manager, this was the biggest cause of trouble. Now it is resolved automatically. However, manual setup may still be needed in environments without internet access.
• Waiting for Elements: driver.get() waits until the page finishes loading, but it does not wait for elements that appear later via Ajax. Using WebDriverWait instead of time.sleep() is a standard practice for professional use.
• Closing (quit): Even if the program stops due to an error, the browser process might remain in the background. Always call driver.quit() within a try...finally block.

Advanced Usage

This example shows how to configure “Headless Mode,” which runs the browser in the background without showing a window. This is used for scheduled tasks on servers or to speed up scraping.

from selenium import webdriver

def run_headless_chrome():
    # Create an options object
    options = webdriver.ChromeOptions()
    
    # Enable headless mode
    options.add_argument("--headless=new")
    
    print("Launching in headless mode (no window will appear)...")
    driver = webdriver.Chrome(options=options)
    
    try:
        driver.get("https://www.google.com")
        print(f"Success: {driver.title}")
    finally:
        driver.quit()

if __name__ == "__main__":
    run_headless_chrome()

Summary

Selenium is a tool that operates a browser via Python code (WebDriver) just like a human would. In the latest versions, manual management of WebDrivers is no longer necessary, and you can start automation immediately with webdriver.Chrome(). Start by trying to open and close the browser.