Here is a list of technologies used for the Krawla Project (in no particular order):

=== python ===


  * python doc: https://docs.python.org


=== zeromq ===


  * brokerless messaging
  * PUSH and PULL should not be used over the internet
  * zeromq guide for python: http://zguide.zeromq.org/py:all
  * doc for pyzmq: https://pyzmq.readthedocs.io/en/latest/


=== mongo ===


  * mongo doc: https://docs.mongodb.com/
  * pymongo doc: https://api.mongodb.com/python/current/


=== json ===


  * json doc: https://www.json.org/


=== msgpack ===


  * msgpack doc: https://msgpack.org/index.html
  * binary serialization protocol
  * faster than json
  * good for data that needs to be sent over the wire
  * bad for files that a human needs to read


=== sqlite ===


  * sqlite doc: https://www.sqlite.org/index.html
  * python sqlite: https://docs.python.org/3/library/sqlite3.html


=== flask ===


  * flask doc: http://flask.pocoo.org/


=== beautifulsoup ===


  * beautifulsoup doc: https://www.crummy.com/software/BeautifulSoup/bs4/doc/


=== css selectors ===


  * css selectors doc: https://www.w3schools.com/cssref/css_selectors.asp


=== regular expressions (regex) ===


  * python doc: https://docs.python.org/3/library/re.html


=== requests ===


  * requests: http://docs.python-requests.org/en/master/


=== firefox ===


  * firefox download: https://www.mozilla.org/en-US/firefox/new/


=== selenium ===


  * selenium: https://github.com/SeleniumHQ/selenium


=== stem ===


  * stem: https://stem.torproject.org/


=== git ===


  * [[git|git]]
  * git doc: https://git-scm.com/docs


=== virtualenv ===


  * virtualenv


=== ssh ===


  * ssh


=== pandas ===


  * [[pandas|pandas]]
  * official documentation: https://pandas.pydata.org/pandas-docs/stable/
  * youtube channel: https://www.youtube.com/user/dataschool/playlists