python - Write text file to pipeline -

-

i have multiple spiders in single scrapy project.

i want write separate output text file each spider spider name , time stamp.

when had single spider creating file in __init method trying this, upromise generate 2 output files while other one. 

class mallcrawlerpipeline(object):  def spider_opened(self, spider):     self.awriter = csv.writer(open('../%s_%s.txt' % (spider.name, datetime.now().strftime("%y%m%d_%h%m%s")), 'wb'),         delimiter=',', quoting=csv.quote_minimal)     self.awriter.writerow(['mall', 'store', 'bonus', 'per_action', 'more_than','up_to', 'deal_url', 'category'])      if 'upromise' in spider.name:         self.cwriter = csv.writer(             open('../%s_coupons_%s.txt' % (spider.name, datetime.now().strftime("%y%m%d_%h%m%s")), 'wb'),             delimiter=',', quoting=csv.quote_minimal)         self.cwriter.writerow(['mall', 'store', 'bonus', 'per_action', 'more_than','up_to', 'deal_url', 'category'])  def process_item(self, item, spider):     self.awriter.writerow([item['mall'], item['store'], item['bonus'], item['per_action'],                            item['more_than'], item['up_to'], item['deal_url'], item['category']])      return item

but facing bug:

 file "c:\users\akhter\dropbox\akhter\mall_crawler\mall_crawler\pipelines.py", line 24, in process_item     self.awriter.writerow([item['mall'], item['store'], item['bonus'], item['per_action'], exceptions.attributeerror: 'mallcrawlerpipeline' object has no attribute 'awriter'

any appreciated. in advance.

are sure you're running obj.spider_opened(...) before obj.process_item(...)? seems you're not, after first method call attribute should have been added object.

if first method call needed perhaps makes sense move __init__ or call there, way.


Comments

Post a Comment

Popular posts from this blog

jasper reports - Fixed header in Excel using JasperReports -

-
how can create excel report fixed header using japserreports? mean need header fixed when scroll excel file. this possible adding couple properties in jrxml file each report. take @ advanced excel features freeze panes. if wanted freeze after first column header (down left side basically) this: <statictext> <reportelement style="sans_bold" mode="opaque" x="0" y="60" width="104" height="20" forecolor="#ffffff" backcolor="#666666"> <property name="net.sf.jasperreports.export.xls.auto.filter" value="start"/> <property name="net.sf.jasperreports.export.xls.column.width" value="110"/> <property name="net.sf.jasperreports.export.xls.freeze.column.edge" value="left"/> </reportelement> &
Read more

media player - Android: mediaplayer went away with unhandled events -

-
i need duration of audio file series of voice announcements need play app. have added audio files resources , play fine. sample code below works perfect intended purpose: return duration of audio files. here code: float getdurationofaudioresource(locationenum loc, context context){ float duration = 0; try { mediaplayer mp; mp = mediaplayer.create(context, getaudioresource(loc)); duration = mp.getduration(); mp.release(); mp = null; } catch (illegalstateexception e) {e.printstacktrace(); logerror(25, "testdescitem:fault::could not open mediaplayer object audio resource.");} return duration; } here's weird thing. code called in main activity prepares set of audio instructions given test. there no errors within activity. second activity called, long string of errors on logcat. 03-07 13:23:43.820: i/actionlogger(21435): gentest_info_test #0 created. 03-07 13:23:43.830: i/actionlogger(21435): gente
Read more

python - ('The SQL contains 0 parameter markers, but 50 parameters were supplied', 'HY000') or TypeError: 'tuple' object is not callable -

-
import pyodbc,nltk,array cnxn = pyodbc.connect('driver={mysql odbc 5.1 driver};server=127.0.0.1;port=3306;database=information_schema;user=root; password=1234;option=3;') cursor = cnxn.cursor() cursor.execute("use collegedatabase ;") cursor.execute("select * sampledata ; ") cnxn.commit() s=[] j=[] x=[] words = [] w = [] sfq = [] pos=[] entry in cursor: s.append(entry.injury_type),j.append(entry.injury_desc) nltk.tokenize import punktwordtokenizer nltk.corpus import stopwords tokenizer = punktwordtokenizer() english_stops = set(stopwords.words('english')) in range(0,26): # filter stop words tokenizer.tokenize(j[i]) w.append([word word in tokenizer.tokenize(j[i]) if word not in english_stops]) in range(0 , 26):#converting tokenzied text ito string sfq.append(" ".join(w[a])) replacers import regexpreplacer replacer = regexpreplacer() in range (0,26):#pos tagging replacer.repl
Read more