As of the 1st of February 2015 this wiki has been deprecated.
This archive is now a read only site.
There is also a copy of the history that has been converted into a git repository at: https://github.com/IATI/IATI-Wiki-Archive.
Please do not rely on any of the information found here.

Linked Data on 4Store

This page provides details of how to install the 4Store triple store, configure it, and post IATI data into it.

Detailed Description

Documentation

This guide is based on starting with a blank Ubuntu 10 LTS Turnkey Core server image. This can be run in a Virtual machine or cloud hosting. The instructions may work for other Ubuntu systems, but will need adapting for different flavours of linux.

Documentation: Configuring 4Store

Install 4Store

apt-get install build-essential libpcre3-dev librasqal2-dev libtool libraptor1-dev libglib2.0-dev ncurses-dev libreadline-dev automake uuid-dev libavahi-core-dev libavahi-glib-dev libavahi-client-dev avahi-daemon
  • Install Raptor2 and RASQL from source
wget http://download.librdf.org/source/raptor2-2.0.8.tar.gz
tar xzf raptor2-2.0.8.tar.gz; rm raptor2-2.0.8.tar.gz
cd raptor2-2.0.8/
apt-get install yajl-tools
./configure
make
make check
make install
ldconfig

wget http://download.librdf.org/source/rasqal-0.9.29.tar.gz
tar xzf rasqal-0.9.29.tar.gz
rm rasqal-0.9.29.tar.gz
cd rasqal-0.9.29.tar.gz
./configure
make
make install
  • Checkout the 4store code from Git
git clone https://github.com/garlik/4store.git
  • Build 4store
cd 4store
sh autogen.sh
./configure
make
make install

Setup Store

Setup a backend database for IATI data

4s-backend-setup iati

Launch the backend

4s-backend iati

Set up a HTTP server for it:

4s-httpd iati -p 8080

Security Note: Check that you have a firewall that only permits access to certain ports in place in front of the server you are running, otherwise any user can post data to your triple store. The TurnkeyHub Core image comes pre-configured with an EC2 firewall.

Setup Apache for Reverse Proxy & Access Control

Install apache and enable reverse proxy

apt-get install apache2
a2enmod proxy_http

Configure /etc/apache2/sites-available with the following configuration

<VirtualHost *:80>

ServerAdmin webmaster@localhost

ProxyRequests Off
ProxyPreserveHost On
<Proxy *>
    Order allow,deny
    Allow from all
</Proxy>

ProxyPass /status http://localhost:8080/status
ProxyPassReverse /status http://localhost:8080/status
ProxyPass /test http://localhost:8080/test
ProxyPassReverse /test http://localhost:8080/test
ProxyPass /sparql http://localhost:8080/sparql
ProxyPassReverse /sparql http://localhost:8080/sparql

#Logging
ErrorLog /var/log/apache2/error.log

# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn

CustomLog /var/log/apache2/access.log combined

</VirtualHost>
  • Restart Apache (and if necessary allow port 80 through on your firewall)
/etc/init.d/apache2 restart

It should now be possible to access /status/ etc. on the server, but not to make post requests to /data/, preventing third parties adding data to the server.

Set 4Store to run on launch

Add the following to /etc/rc.local

export PATH='/root/Jena/bin:/usr/lib/git-core:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
4s-backend iati
4s-httpd -p 8080 iati

Add 4store to backups

If using the Turnkey Linux backup manager (tklbam) then you need to explicitly add the Turnkey directory to the backup with:

echo /var/lib/4store >> /etc/tklbam/overrides

Documentation: Loading data

You can load data to 4Store using the IATISyncTools. You will need the Sync Tools and the IATI-XSLT templates

cd ~/
mkdir scripts
cd scripts
git clone https://github.com/aidinfolabs/IATISyncTools
git clone https://github.com/aidinfolabs/IATI-XSLT.git

Then switch to the data where you want to store IATI data:

cd ~/
mkdir data
cd data

And run the iati_download script. (Run with –help option to see all parameters)

python ~/scripts/IATISyncTools/download/iati_download.py -v3

Then change to the directory of files you want to convert to linked data:

python ~/scripts/IATISyncTools/conversions/linked_data/convert.py

This will generate RDF files for each one. Then run:

python ~/scripts/IATISyncTools/conversions/linked_data/upload.py

If you triple store settings are different from those described above you may need to modify this script.


QR Code
QR Code Linked Data on 4Store (generated for current page)