python - Scrapy ERROR: Spider must return Request, BaseItem or None, got 'dict' -
i have tried reproducing scrapy tutorial using xpath , keep running error: spider must return request, baseitem or none, got 'dict' in <get http://quotes.toscrape.com/>
not sure how fix this.
i'm going share snippets 2 files should enough debugging:
1) spider quotes_spider.py
from scrapy.spider import spider scrapy import request class quotespider(spider): name = 'quotes' start_urls = [ 'http://quotes.toscrape.com/', ] def parse(self, response): quote in response.xpath('//div[@class="quote"]'): yield { 'text': quote.xpath('.//span[@class="text"]/text()').extract(), 'author': quote.xpath('.//small[@class="author"]/text()').extract(), 'tags': quote.xpath('.//div[@class="tags"]/a[@class="tag"]/text()').extract(), }
2) items.py
from scrapy.item import item class quotesbotitem(item): text = scrapy.field() author = scrapy.field() tags = scrapy.field()
fyi: in case compare tutorial , wondering why switched extract_first()
extract()
, it's because seeing error exceptions.attributeerror: 'selectorlist' object has no attribute 'extract_first'
unrelated question believe.
you returning dictionary error says , not item
class quotespider(spider): name = 'quotes' start_urls = [ 'http://quotes.toscrape.com/', ] def parse(self, response): quote in response.xpath('//div[@class="quote"]'): item = quotesbotitem() item['text'] = quote.xpath('.//span[@class="text"]/text()').extract() item['author'] = quote.xpath('.//small[@class="author"]/text()').extract() item['tags'] = quote.xpath('.//div[@class="tags"]/a[@class="tag"]/text()').extract() yield item
Comments
Post a Comment