Python Tutorial: How to Parse and Combine RSS News headlines using feedparser

This python code will take a list of RSS newsfeed urls, fetch and combine all the news headlines into one list.  The feedparser package is required, install it with the following command.

pip install feedparser

import feedparser

# Function to fetch the rss feed and return the parsed RSS
def parseRSS( rss_url ):
    return feedparser.parse( rss_url ) 
    
# Function grabs the rss feed headlines (titles) and returns them as a list
def getHeadlines( rss_url ):
    headlines = []
    
    feed = parseRSS( rss_url )
    for newsitem in feed['items']:
        headlines.append(newsitem['title'])
    
    return headlines

# A list to hold all headlines
allheadlines = []

# List of RSS feeds that we will fetch and combine
newsurls = {
    'apnews':           'http://hosted2.ap.org/atom/APDEFAULT/3d281c11a76b4ad082fe88aa0db04909',
    'googlenews':       'http://news.google.com/?output=rss',
    'yahoonews':        'http://news.yahoo.com/rss/'
}

# Iterate over the feed urls
for key,url in newsurls.items():
    # Call getHeadlines() and combine the returned headlines with allheadlines
    allheadlines.extend( getHeadlines( url ) )


# Iterate over the allheadlines list and print each headline
for hl in allheadlines:
    print(hl)
 

# end of code

Output


France bombs Islamic State HQ, hunts attacker who got away
Clinton campaign defends debate 9/11 remarks
Donald Trump: ‘O’Malley is a clown’; ‘Hillary is owned by Wall Street’
College student from California studying abroad killed in Paris attacks
About 1,500 Mormons resign from church in protest of same-sex policy
Pentagon says five Guantanamo detainees transferred to United Arab Emirates
Paris attacks: Video shows firefight outside Bataclan
Paris attacks show U.S. surveillance of Islamic State may be ‘going dark’
Belgian connection: At least 3 held in Brussels over Paris attacks
Clinton wobbled on foreign policy in debate
Clinton cites 9/11 in defending Wall Street donations
Sanders scores applause for Eisenhower quip
Sanders campaign claims victory in CBS dispute
Paris attacks may lead to US military anti-IS escalation
Sanders aide pushes back against CBS switch to foreign policy focus for debate
Five questions about Paris for Clinton, Sanders, and O’Malley
France Strikes ISIS Targets in Syria in Retaliation for Attacks - New York Times
Clinton's debate performance leaves trail of fodder for political adversaries - Washington Post
Parisians united: What the attacks mean to us - CNN
In wake of attacks, presidential contenders focus on Syrian refugees - Miami Herald
The Belgian neighborhood indelibly linked to jihad - Washington Post
Americans pay respects to Paris terror victims - USA TODAY
Pentagon transfers 5 Yemenis being held at Guantanamo Bay to UAE - Washington Post
Paris unites in defiant solidarity, then scatters in panic - Washington Post
Paris terror attack: Names of victims start to emerge - CNN
Patriots still undefeated after late field goal - NFL.com


Download this code as a IPython Notebook

I hope you find this python example useful and educational. You are free to use the above code how you see fit. I do however suggest that you implement some type of rss feed caching as some services may block your ip for excessive requests.

More Python Tutorials

The post Python Tutorial: How to Parse and Combine RSS News headlines using feedparser appeared first on jcutrer.com.