Extract Unique Item Titles with Scrapy

This function extracts unique item titles from a Scrapy spider's response. It uses CSS selectors to locate elements and a set to ensure the uniqueness of the titles.

Technology Stack : Scrapy

Code Type : Scrapy crawler handler

Code Difficulty : Intermediate

                
                    
def extract_unique_items(spider, response):
    # Extract unique items from the response using scrapy's built-in functions
    unique_items = set()
    for item in response.css('div.item'):
        unique_items.add(item.css('span.title::text').get())
    return list(unique_items)