proxy - Python requests proxies.txt IP:Port:User:pass -
i trying use file store proxies in it. want load these on proxies on requests module.
but problem is, proxies mixed up. proxies authentication, , doesn´t.
so proxies.txt looks this
123.12.190.121:2323:user:pass 123.12.190.122:2323:user:pass 123.12.190.123:2323 123.12.190.124:2323:user:pass 123.12.190.125:2323
some authentication, doesn´t.
now want call command this
response = session.get(url, proxies=proxies)
with loaded proxies stored in proxies.txt
can me out?
from requests documentation can use proxy user/pass or without. please check following link: http://docs.python-requests.org/en/master/user/advanced/
to use http basic auth proxy, use http://user:password@host/ syntax:
proxies = {'http': 'http://user:pass@10.10.1.10:3128/'}
so need create proxies dictionary or regular expression or splitting ':' character. can create proxy url like: http://user:pass@10.10.1.10:3128 or http://10.10.1.10:3128.
from documentation: give proxy specific scheme , host, use scheme://hostname form key. match request given scheme , exact hostname.note proxy urls must include scheme.
proxies = {'http://10.20.1.128': 'http://10.10.1.10:5323'}
please check code snippet:
import re import random scheme = 'http://' proxies = {} open('proxy.txt', 'r') f: line in f: pr = line.strip() m = re.search(r'^(\d+\.\d+\.\d+\.\d+)\:(\d+)\:([^\:]*)\:([^\$]*)$', pr) if m: print('user: ' + m.group(3)) proxies[scheme + m.group(1)] = scheme + m.group(3) + ':' + m.group(4) + '@' + m.group(1) + ':' + m.group(2) else: proxies[scheme + pr] = scheme + pr print(proxies) k = random.choice(list(proxies)) print(k + ' => ' + proxies[k])
Comments
Post a Comment