This archive is now a read only site.
There is also a copy of the history that has been converted into a git repository at: https://github.com/IATI/IATI-Wiki-Archive.
Please do not rely on any of the information found here.
Table of Contents
Aid Information Toolkit
Linked to work on the idea of a Public Good Infrastructure we have been exploring the idea of creating a distribution of open source tools that all work together to perform key data access and use functions.
A full outline of the 'Open Aid Information Toolkit' idea can be found in this document.
This page can be used to start scoping out that work.
Desirable features / Architecture
What features would be desirable in a set of tools.
A powerful and interoperable set of scripts
- Able to run across platforms
- Shared configuration (e.g. for working directories etc.)
- Able to be connected together as a pipeline of tools
- Command line tools;
- Open source;
- Accessible web-based front end
- Cloud version available
- Easy to install and deploy
- Should make it possible to understand what has happened to data along a processing pipeline
Registry download tool
For fetching any changed packages for a given query.
For validating IATI XML files.
Should be able to:
- Check for well formed data
- Validate against the schema
- Check for codelist values
Written up for RHOK Soton
This component should be able to read in any IATI XML file and convert transaction and budget values from one currency to another. This requires:
- An open list of historic currency conversion rates - no comprehensive data exists, but it should be possible to use one or more of the datasets on the datahub to generate reasonably granular conversion tables for most major currencies.
- API based solutions may be possible, but a large conversion process could involve thousands of transactions, exceeding the limits of most free APIs.
- It would be useful to have scripts that automatically update conversion tables
- A process to read in IATI XML files and to work out the current currency and value-date of any transaction / budget
- Currencies may be provided in a @default-currency attribute of an iati-activity, or at the level of an individual transaction value
- A process to write the converted values back to the file
- Ideally there would be the option to either overwrite the current currency values, changing default / element level currency in the process or to add a duplicate transaction in the new currency.
- It would be useful to be able to run these conversions in parallel: for example, as a map-reduce job, to be able to convert 100s of files rapidly (there are over 1500 IATI files now, and thousands more anticipated in coming months; some processes run overnight updates of the data, and being able to do this rapidly would be beneficial).
- We need environment variables that can be shared across scripts
- We need a shared logging framework
Consider a model with a master script and then a set of plugins. Scripts would be executed from the master script to allow use of the same logging framework etc.
We need a light-weight structure to manage a community of contributors. Consider learning from the Open Source community.
Links, source code and development plans
See http://openhatch.org/ for ideas around supporting open source communities.