S3QL is an online file system for UNIX-like operating systems that provide features like compression, encryption, data de-duplication, immutable trees and snapshotting. It can be used to create a local or remote (online) file system and so can be used to backup your data to a supported Cloud storage platform.
Google Storage, Amazon S3 and OpenStack are supported online Cloud storage services. There are two branches of S3QL – S3QL-1.x and 2.x. The latter being the current active branch, with more features than the former. The 1.x branch does not get new features, only bugfixes. The other difference between the two branches is that the 1.x branch is compatible with Python 2.7, while the 2.x branch is compatible with Python 3.3.
If you run yum install s3ql on Fedora 19, you will be installing a 1.x version, because there is no binary package for a 2.x version in the repository. Which means that to install S3QL-2.x on Fedora 19 and, therefore, get access to the latest features, you will have to compile and install it from source. This article shows how to do just that – compile and install the latest version of S3QL on Fedora 19.
Before you can start compiling the package, here are the list of dependencies that must be installed first:
- GCC. Installed using yum install gcc
- Python 3 and Python 3 development packages. You can install these by typing yum install python3 python3-devel
- Python 3 setup tools, which can be installed using yum install python3-setuptools
- Python3-apsw. Installed using yum install python3-apsw
- Libattr-devel. Installed using yum install libattr-devel
- Pycryptopp. Installed using yum install pycryptopp
- Pycrypto. Installing Pycryptopp does not appear to satisfy this dependency, so this has to be installed. However, because there is no binary package in the repository, it has to be compiled from source. To do that, follow these steps:
- Download it using wget http://ftp.dlitz.net/pub/dlitz/crypto/pycrypto/pycrypto-2.6.tar.gz
- Uncompress and untar it using tar zxf pycrypto-2.6.tar.gz
- Change into the directory (cd pycrypto-2.6) and install it using python3 setup.py install
- Python-paramiko. Installed using yum install python-paramiko
- xz-devel. Installed using yum install xz-devel
- Fuse and Fuse development packages. Installed using yum install fuse fuse-devel
- llfuse. This dependency has to be compiled from source because the version of the binary package available in Fedora (llfuse-0.38) is lower than the required version (llfuse-0.39). If you installed all the dependencies, and opt to use the binary package in the repository, you will get this error when compiling S3QL:
pkg_resources.VersionConflict: (llfuse 0.38 (/usr/lib64/python3.3/site-packages), Requirement.parse(‘llfuse>=0.39’))
To compile version 0.39 from source, follow these steps:
- Download it using wget http://python-llfuse.googlecode.com/files/llfuse-0.39.tar.bz2
- Then uncompress and untar it by typing bunzip2 -d llfuse* && tar xf llfuse*
- Change into the directory (cd llfuse*) and install it using python3 setup.py install
- SQLite and its development packages. Installed using yum install sqlite sqlite-devel
After all the dependencies have installed successfully, you may now begin the process of compiling S3QL.
- Download it with wget http://s3ql.googlecode.com/files/s3ql-2.4.tar.bz2
- Unpack it with bunzip2 -d s3ql-2.4.tar.bz2 && tar xf s3ql-2.4.tar
- Change into the directory (cd s3ql-2.4) and run
python3 setup.py build_ext --inplaceto build it - Then python3 runtests.py tests to run a self-test. During my test installation, most of the tests were either SKIPPED or PASSED. There was an error related to SELinux. Part of the output is below:
rsync: rsync_xal_set: lsetxattr(“”/tmp/s3ql-mnt-ssn0wz/orig/C!pulseaudio/Oõkarm/.sort.B3KkHI””,”security.selinux”) failed: Operation not supported (95)
rsync: rsync_xal_set: lsetxattr(“”/tmp/s3ql-mnt-ssn0wz/orig/C!pulseaudio/Oõkarm/.ótrue.WWN7XB””,”security.selinux”) failed: Operation not supported (95)
rsync: rsync_xal_set: lsetxattr(“”/tmp/s3ql-mnt-ssn0wz/orig/C!pulseaudio/Oõkarm/.Ðkinfocenter.lPArfv””,”security.selinux”) failed: Operation not supported (95)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1052) [sender=3.0.9]
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: stopping after 2 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=============== 1 failed, 135 passed, 61 skipped, 1 error in 109.11 seconds======== - Even with this SELinux-related error, running python3 setup.py install completed successfully
Now that S3QL is installed, the next article will show to to use it to encrypt and backup your data online. Meanwhile, you may read the user guide here.
