python - Access Data from Behind Java -

-

im trying extract goal times dropdown on each fixture here http://www.bbc.co.uk/sport/football/league-one/results

i can't seem find data when searching - ideas why?

import requests bs4 import beautifulsoup  # load page data r = requests.get("http://www.bbc.co.uk/sport/football/league-one/results") soup = beautifulsoup(r.content) print soup.prettify()  # save teams link in soup.find_all("a"):     print link.text  # save results link in soup.find_all("abbr"):     print link.text

ff

so data heavy (not mention slow because of loading separate pages) , may block excess amount of requests, way see doing it. go through , href associated results button, load page , parse score information it

import requests bs4 import beautifulsoup  def parse_page(data):         subsoup = beautifulsoup(data)         matchoverview = subsoup.find('div', attrs={'id':'match-overview'})         print '--------------'         hometeam = matchoverview.find('div', attrs={'class':'team-match-details'}).findnext('span').findnext('a').text         homescore = matchoverview.find('div', attrs={'class':'team-match-details'}).findnext('span').findnext('span').text         homegoalscorers = ["home goal scorers:"]         goals in matchoverview.find('div', attrs={'class':'team-match-details'}).findnext('p').find_all('span'):             homegoalscorers.append(goals.text.replace(u'\u2032', "'"))         homegoals = "\n".join(homegoalscorers)         awayteam = matchoverview.find('div', attrs={'id': 'away-team'}).find('div', attrs={'class':'team-match-details'}).findnext('span').findnext('a').text         awayscore = matchoverview.find('div', attrs={'id': 'away-team'}).find('div', attrs={'class':'team-match-details'}).findnext('span').findnext('span').text         awaygoalscorers = ["away goal scorers:"]         goals in matchoverview.find('div', attrs={'id': 'away-team'}).find('div', attrs={'class':'team-match-details'}).findnext('p').find_all('span'):             awaygoalscorers.append(goals.text.replace(u'\u2032', "'"))         awaygoals = "\n".join(awaygoalscorers)         print '{0} {1} - {2} {3}'.format(hometeam, homescore, awayteam, awayscore)         print homegoals         print awaygoals  def all_league_results():     r = requests.get("http://www.bbc.co.uk/sport/football/league-one/results")     soup = beautifulsoup(r.content)      # save teams     link in soup.find_all("a", attrs={'class': 'report'}):         fulllink = 'http://www.bbc.com' + link['href']         subr = requests.get(fulllink)         parse_page(subr.text)  def specific_game_results(url):     subr = requests.get(url)     parse_page(subr.text)  #get specific games results specific_game_results('http://www.bbc.co.uk/sport/0/football/32460049') #get current league results all_league_results()

Comments

Post a Comment

Popular posts from this blog

python - pip install -U PySide error -

-
does know how avoid following error when running pip install -u pyside per official website instructions here: https://pypi.python.org/pypi/pyside/#installing-pyside-on-a-mac-os-x-system note have done brew install qt successfully. you using pip version 7.0.3, version 7.1.2 available. should consider upgrading via 'pip install --upgrade pip' command. collecting pyside using cached pyside-1.2.4.tar.gz installing collected packages: pyside running setup.py install pyside complete output command /applications/anaconda/bin/python -c "import setuptools, tokenize;__file__='/private/var/folders/dy/ttgqhqqx3g9bsbsnqnxsk2z80000gq/t/pip-build-cpxemt/pyside/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/dy/ttgqhqqx3g9bsbsnqnxsk2z80000gq/t/pip-0r7hri-record/install-record.txt --single-version-externally-managed --compile: ...
Read more

arrays - C++ error: a brace-enclosed initializer is not allowed here before ‘{’ token -

-
simple question: i'm trying initialize array within c++ class declaration: using namespace std; #include <string> class myclass{ public: string myarray[] = {"a","b","c"}; }; and i'm getting error: error: brace-enclosed initializer not allowed here before ‘{’ token no, without complied c++11 compiler, cannot initialize member array in declaration. have initialize array member in constructor. , don't use open array if know number of elements initialize array.
Read more

apache - setting document root in antoher partition on ubuntu -

-
i place apache document root inside partition of ubuntu hard drive, keep getting forbidden message, when place home directory woking find, how be? group or owner affected ? here mysite.conf , apache2.conf when place document root in home folder (working) #site-available/mysite.conf documentroot /home/jono/www #/etc/apache2/apache2.conf <directory /home/jono/www/> options indexes followsymlinks allowoverride none require granted </directory> bu when change document root partition keep getting forbidden messasge #site-available/mysite.conf documentroot /media/jono/website_data/www #/etc/apache2/apache2.conf <directory /media/jono/website_data/www/> options indexes followsymlinks allowoverride none require granted </directory> is owner/group access affected ? or there problem ? at last it's working, have grant access www:data accesing whole directory @mark-b, using chown -r www-data:www-data whole directory is...
Read more