728x90 반응형 BIG DATA43 네이버 환율 데이터 가져오기 - css 셀렉터를 안쓰는 방식 from bs4 import BeautifulSoup as BS import requests as req # iframe 안에 들어있는 표 값. url = "https://finance.naver.com/marketindex/exchangeList.naver" res = req.get(url) soup = BS(res.text, "html.parser") #print(soup.title) #print(soup.title.string) tds = soup.find_all("td") #print(tds) names = [] for td in tds: #print(td.string) if len(td.find_all("a")) == 0: continue #print(td.string) # 이러면 공백이 많다... 2021. 9. 24. Top 11 FREE Web Scraping Frameworks (2019년글) https://geekflare.com/web-scraping-frameworks/ 1. Scrapy 2. MechanicalSoup 3. Jaunt 4. Jauntium 5. Storm Crawler 6. Norconex 7. Apify 8. Kimurai 9. Colly 10. Grablab 11. BeautifulSoup 2021. 9. 10. 12 Best Web Scraping Tools https://www.hdfstutorial.com/blog/best-web-scraping-tools/ 툴 소개 1. Luminati Data Collector 2. ProWebScrapper 3. webscraper.io 4. data-miner.io 5. scrapy.org 6. scraper 7. webhose.io 8. OutWit.com 9. FMiner.com 10. PySpider 11. Apify SDK 12. Parsehub.com 2021. 9. 10. c++ comsumper 개발환경 구축 참고 : 1. https://docs.confluent.io/clients-librdkafka/current/overview.html 개발환경 설정 1. 라이브러리 인스톨 sudo apt-get install librdkafka-dev 2. Makefile 설정 관련 헤더파일 위치 : /usr/include/librdkafka rdkafka.h rdkafkacpp.h 라이브러리파일 위치 : /usr/lib/x86_64-linux-gnu librdkafka++.a librdkafka++.so librdkafka++.so.1 librdkafka.a librdkafka.so librdkafka.so.1 MYSQLINCDIR = /usr/include/mysql KAFKAINCDIR = /usr/include/.. 2021. 6. 14. 이전 1 ··· 7 8 9 10 11 다음 728x90 반응형