- XGrid - a distributed computing system from Apple. (if you have a Mac and want to contribute with some spare cycles to this project please contact us !)
Theseus
Theseus is a python package developed by The Observatorium team, including several modules to deal with webpage retrival and text processing. It includes python scripts that, together with bash scripts, are used for collecting and processing web pages for the observatorium databases.
Theseus is and open source library free for research usage. Please access the Theseus website, feel free to download and try it, and contact us if necessary.
Other software
Other software that we are using for processing large data: