- XGrid - a distributed computing system from Apple. (if you have a Mac and want to contribute with some spare cycles to this project please contact us !)
Theseus is a python package developed by The Observatorium team, including several modules to deal with webpage retrival and text processing. It includes python scripts that, together with bash scripts, are used for collecting and processing web pages for the observatorium databases.
Other software that we are using for processing large data: