我正在尝试从CNN文章中提取元标记
import httplib2
from bs4 import BeautifulSoup
http = httplib2.Http()
status, response = http.request(http://www.cnn.com/2016/08/09/health/chagas-sleeping-sickness-leishmaniasis-drug/index.html)
soup = BeautifulSoup(response)
print(soup.select('body > div.pg-right-rail-tall.pg-wrapper.pg__background__image > article > meta'))
我正在尝试将其缩小到仅此输出
<meta content="health" itemprop="articleSection"><meta content="2016-08-09T12:10:24Z" itemprop="dateCreated"><meta content="2016-08-09T12:10:24Z" itemprop="datePublished"><meta content="2016-08-09T12:10:24Z" itemprop="dateModified"><meta content="http://www.cnn.com/2016/08/09/health/chagas-sleeping-sickness-leishmaniasis-drug/index.html" itemprop="url"><meta content="Meera Senthilingam, for CNN" itemprop="author"><meta content="Could one discovery take on three deadly parasites? - CNN.com" itemprop="headline"><meta content="Three seemingly different diseases infect 20 million people each year: Chagas disease, leishmaniasis and African sleeping sickness. But one drug could be developed to fight all three." itemprop="description"><meta content="sleeping sickness, disease, drug, drug development, chagas disease, leishmaniasis, Novartis, health, Could one discovery take on three deadly parasites? - CNN.com" itemprop="keywords"><meta content="http://i2.cdn.turner.com/cnnnext/dam/assets/150812101743-chagas-bug-large-tease.jpg" itemprop="image"><meta content="http://i2.cdn.turner.com/cnnnext/dam/assets/150812101743-chagas-bug-large-tease.jpg" itemprop="thumbnailUrl"><meta content="Could one discovery take on three deadly parasites? " itemprop="alternativeHeadline">
但由于某种原因,该BeautifulSoup.select()
方法返回的HTML大约是我想要的100倍。对于如何解决此问题,我将不胜感激。
问题是解析器/ html,lxml和html5lib为您提供了您想要的东西。
soup = BeautifulSoup(response,"lxml")
或者:
soup = BeautifulSoup(response,"html5lib")
如果您有没有做LXML或html5lib安装,您可以安装html5lib使用PIP,LXML是根据您的操作系统,因为它有几个依赖多一点参与,但它绝对是值得安装。
您还可以简化选择:
soup.select('div.pg-right-rail-tall.pg-wrapper.pg__background__image meta')
本文收集自互联网,转载请注明来源。
如有侵权,请联系[email protected] 删除。
我来说两句