python for android : BeautifulSoup 有 bug
时间:2014-05-03 17:33:08
收藏:0
阅读:427
BeautifulSoup 善于网页数据分析 ,但是 python for android : BeautifulSoup 有 bug ,
text = h4.a.text 只能取得 None,所以我写了function: getText()
来fix this bug.
例如: 抓取CSDN极客头条内容 soup.py
import urllib2, re from BeautifulSoup import BeautifulSoup import sys reload(sys) sys.setdefaultencoding(‘utf-8‘) def getText(text): begin = text.find(‘>‘,0) if begin > -1: begin += 1 end = text.find(‘</a>‘,begin) if begin < end: return text[begin:end].strip() else: return None else: return None page = urllib2.urlopen("http://geek.csdn.net/new") soup = BeautifulSoup(page) for h4 in soup.findAll(‘h4‘): if h4.a is not None: href = h4.a.get(‘href‘) text = getText(str(h4.a)) print text print href page.close()
请参考: http://www.crummy.com/software/BeautifulSoup/bs3/documentation.zh.html
评论(0)