Sorry, we don't support your browser.  Install a modern browser

Focus on main content#166

?

Focus on main content: Very frequently the results that you show are based on (a) text in ads on the site or (b) related articles, but not the main article itself.
Ads should obviously be excluded and if a related article exists for that term that article should be referenced rather than the one that deals with something else (i.e. either discarding the wrong main article or sending your bot to scrape the related article that deals with the keyword). In order to accomplish this, you could apply an approach that services such as Instapaper or Pocket apply, which only parse the main body content itself rather than all the other noise-generating content around it.

3 months ago