web scraping - Loop through payload Python -


there site connect to, need login 4 times different user names , passwords.

is there anyway can looping through usernames , passwords in payload.

this first time im doing , not sure of how go it. code works fine if post 1 username , password.

im using python 2.7 , beautifulsoup , requests.

here code.

import requests import zipfile, stringio bs4 import beautifulsoup  # here add login details submitted login form. payload = [ {'username': 'xxxxxx','password': 'xxxxxx','option': 'login'}, {'username': 'xxxxxx','password': 'xxxxxxx','option': 'login'}, {'username': 'xxxxx','password': 'xxxxx','option': 'login'}, {'username': 'xxxxxx','password': 'xxxxxx','option': 'login'}, ] #possibly need headers later. headers = {'user-agent': 'mozilla/5.0 (macintosh; intel mac os x 10_12_5) applewebkit/537.36 (khtml, gecko) chrome/59.0.3071.115 safari/537.36'} base_url = "https://service.rl360.com/scripts/customer.cgi/sc/servicing/"  requests.session() s:         p = s.post('https://service.rl360.com/scripts/customer.cgi?option=login', data=payload)          # download page scrape.         r = s.get('https://service.rl360.com/scripts/customer.cgi/sc/servicing/downloads.php?folder=datadownloads&sortfield=expirydays&sortorder=ascending', stream=true)         content = r.text         soup = beautifulsoup(content, 'lxml')         #now recent download url.         download_url = soup.find_all("a", {'class':'tabletd'})[-1]['href']         #now join base url download url.         download_docs = s.get(base_url + download_url, stream=true)         print "checking content"         content_type = download_docs.headers['content-type']         print content_type         print "checking filename"         content_name = download_docs.headers['content-disposition']         print content_name         print "checking download size"         content_size = download_docs.headers['content-length']         print content_size          #this extract , download specified xml files.         z = zipfile.zipfile(stringio.stringio(download_docs.content))         print "---------------------------------"         print "downloading........."         #now save files specified location.         z.extractall('c:\temp')         print "download complete"

just use loop. may need adjust download directory if files overwritten. 

payloads = [ {'username': 'xxxxxx1','password': 'xxxxxx','option': 'login'}, {'username': 'xxxxxx2','password': 'xxxxxxx','option': 'login'}, {'username': 'xxxxx3','password': 'xxxxx','option': 'login'}, {'username': 'xxxxxx4','password': 'xxxxxx','option': 'login'}, ]  ....  payload in payloads:     requests.session() s:         p = s.post('https://service.rl360.com/scripts/customer.cgi?option=login', data=payload)         ...

Comments

Post a Comment

Popular posts from this blog

python Tkinter Capturing keyboard events save as one single string -

is possible store keyboard events 1 string? code below stores 1 char , prints it. card reader or bar code reader, contains collection of character/ string not 1 character @ time. goal save char pressed text variable. from tkinter import * root = tk() def key(event): text= event.char text+= event.char print ("pressed", text) def callback(event): frame.focus_set() print ("clicked at", event.x, event.y) frame = frame(root, width=100, height=100) frame.bind("<key>", key) frame.bind("<button-1>", callback) frame.pack() root.mainloop() currently, creating text variable , everytime key() function gets called, , text stores last character typed. you can define text module level variable , , use module level text inside key function - text = '' def key(event): global text text+= event.char print("pressed", text)
Read more

android - InAppBilling registering BroadcastReceiver in AndroidManifest -

i'm implementing inapp billing in android app. works well, however, i'm trying decouple broadcast receiver activity manifest. suggestion in android's trivialdrive sample: // important: dynamically register broadcast messages updated purchases. // register receiver here instead of <receiver> in manifest // because call getpurchases() @ startup, therefore can ignore // broadcasts sent while app isn't running. // note: registering listener in activity bad idea, done here // because sample. regardless, receiver must registered after // iabhelper setup, before first call getpurchases(). currently there's class extends broadcastreceiver : public class iabbroadcastreceiver extends broadcastreceiver { /** * intent action receiver should filter for. */ public static final string action = "com.android.vending.billing.purchases_updated"; private final iabbroadcastlistener mlistener; public iabbroadcastreceiver(iabbroadcastlistener listener) { m
Read more

javascript - Z-index in d3.js -

i have map , when hover effect on region of map below yellow line , circle, last ones disappear. how can solve problem? http://jsfiddle.net/ot21n9qx/ var arc = g.append("path") .style("fill", "none") .style("stroke", "yellow") .style("stroke-width", 2) .attr("d", "m" + pathorigin[0] + "," + pathorigin[1] + " q" + svgpoint[0] + "," + pathorigin[1] + " " + svgpoint[0] + "," + svgpoint[1]); var circlesize = d3.scale.linear() .domain([0, 0.5, 1]) .range([4, 10, 4]); svg not have z order draws objects in order created. need create path , circle after create highlight regions. here forked fiddle: http://jsfiddle.net/o00fzgaf/1/
Read more