EIJC17 & Dataharvest

Follow the money - hands-on workshop – finding your way through the data. In the hands-on session we will use the subsidy-story dataset to show how to start finding stories from a dataset that is over 2 million lines big. Participants will learn about the dataset itself, as well as acquire skills to quickly get to know a dataset that is beyond the size of a normal spreadsheet program, running basic statistics and validation strategies that we used. We will first demonstrate the work we did on the hack day for inspiration. This will show what tools can be used to work through a 2 million-line datasets, such as jupiter notebooks, SQL, and the Open Spending toolbox. Starting with the OpenSpending toolbox, we provide a guided tour through the data, adding different filters and answering simple questions by trying different visualisations, and sorting options. In the second part, we already go into advanced mode and take the participants through the SQL-data-miner that is linked to the Open Spending repository and runs on our datasets. We will run basic statistics and some tailored queries together to show how to easily get to know a dataset of this size. Finally, we show how we validated our dataset, walking the participants through different validation strategies we used from comparing it to raw data, to making sure certain variables are always present etc.