python中selenium動態切換ip的問題(如何使每次切換頁面只切換ip但不打開穀歌瀏覽器)

CSDN問答 2022-01-07 09:12:33 阅读数:1,007

python selenium ip 每次 ip
def ip_log(ip,port): PROXY = f"{ip}:{port}" # 這裏寫你的代理 chrome_options = webdriver.ChromeOptions() chrome_options.add_argument('--proxy-server=%s' % PROXY) global browser browser = webdriver.Chrome(executable_path='./chromedriver', options=chrome_options)def data(value): # 執行一組js程序,拉到頁面底部 browser.execute_script('window.scrollTo(0,document.body.scrollHeight)') sleep(2) global shop_name_list, shop_price_list, shop_people_list, shop_location_list,a shop_name_list = [] shop_price_list = [] shop_people_list = [] shop_location_list = [] ip_list = ['36.248.132.187','36.248.132.23','122.4.48.145'] #輸三個代理ip port_list = [9999,9999,9999] #ip對應的port a = 0 b = 44 c = 0 for i in range(1,6): page = browser.page_source soup = BeautifulSoup(page, 'lxml') shop_data_list = soup.find('div', class_='grid g-clearfix').find_all_next('div', class_='items') for shop_data in shop_data_list: # 商品名稱 shop_image_data = shop_data.find_all('div',class_='pic') for shop_data_a in shop_image_data: shop_data_a = shop_data_a.find_all('a',class_='pic-link J_ClickStat J_ItemPicA') for shop_name in shop_data_a: shop_name = shop_name.find_all('img')[0]['alt'] shop_name_list.append(shop_name) # 商品價格 shop_price_data = shop_data.find_all('div',class_='price g_price g_price-highlight') for shop_price in shop_price_data: shop_price_list.append(shop_price.text.strip()) # 付款人數 shop_people_number_data = shop_data.find_all('div',class_='deal-cnt') for shop_people_number in shop_people_number_data: shop_people_list.append(shop_people_number.text) #地址s shop_location_data = shop_data.find_all('div',class_='location') for shop_location in shop_location_data: shop_location_list.append(shop_location.text) # 實現動態加載代理ip if c == 0: ip_log(ip_list[c],port_list[c]) c += 1 if c == 1: ip_log(ip_list[c],port_list[c]) c += 1 if c == 2: ip_log(ip_list[c],port_list[c]) c = 0 shop_data = zip(shop_name_list,shop_price_list,shop_people_list,shop_location_list) for data in shop_data: print(data) a += 1 b += 44 browser.get(f"https://s.taobao.com/search?q={value}&s={b}") sleep(0.5) print('已成功爬取:%s條信息'%a) return shop_name_list, shop_price_list, shop_people_list, shop_location_list,a



采納答案:

題主意思是每次切換時桌面不顯示瀏覽器窗口吧,否則你不打開它,無法獲取數據的,如果想要隱藏瀏覽器界面,在選項中設置,

chrome_options = webdriver.ChromeOptions()

chrome_options.add_argument('--proxy-server=%s' % PROXY)

兩行的後面加上下面一句即可。

chrome_options.add_argument('--headless')#不顯示瀏覽器窗口



其他答案2:

您好,請問解决了嗎,我也想實現這樣子的功能

版权声明:本文为[CSDN問答]所创,转载请带上原文链接,感谢。 https://gsmany.com/2022/01/202201070912325177.html