Simple Data Backup with rsync

I struggled for years to manage simple home data backups effectively but a nice Linux tool (rsync) exists to make it very manageable, here are some use cases I use frequently to make it a breeze:

A straight copy of one drive to another:

  • n – this will show the output without doing anything (dry run), remove this to run the backup.
  • r – this means recursive, basically it will catch all files.
  • u – this skips files that are newer on ‘drive2’, I use this to ensure files are a true copy in case of a mix-up.
  • v – makes the output verbose, gives lots of information on progress.
rsync -nruv /media/drive1/ /media/drive2

As of recent I started using the –checksum argument due to not following my own rules and doing a backup that overwrote modification time of all files, this argument looks at the file checksum (unique identifier) as opposed to the modification time to do a backup:

rsync -nruv --checksum /media/drive1/ /media/drive2

This was supposed to just get you on your feet with backups, a lot more options are available, read on here perhaps:

By modifying a tutorial on the opensource blog I was really able to streamline and speed up the backups, save the below as a shell script, it makes light work of multiple Terabytes of data:


for DIR in $DIRS; do
     cd "$SRC"/$DIRS
     rsync -cdlptgov --delete . /"$DEST"/$DIR
     find . -maxdepth 1 -type d -not -name "." -exec rsync -crlptgov --delete {} /"$DEST"/$DIR \;

Resources I used:

That’s it!

Easy Media Management with exiftool

I struggled for years to manage pictures effectively but a nice Linux tool (exiftool) exists to make it very manageable, here are some use cases I use frequently:

Change all times of all pictures in directory by one hour:
(change number/sign for any other hours, it is clever and will adjust day if crosses midnight etc.)

$ exiftool -AllDates+=1 -overwrite_original *

Remove all EXIF metadata from images with “.jpeg” extensions only:

exiftool -all= *.jpeg

Add if statements to operations if required: (for example only adjust picture taken with Canon Cameras)

$ exiftool -AllDates+=1 -overwrite_original -if '$make eq "Canon"' -r *

Some other rename commands that help with naming if required:

Cuts start of filename: (by 4 letters, adjust as required) (-n = dry run)

$ rename -n -v  's/^(.{4})//' *

Changes file extensions from upper case to lower case.

rename 'y/A-Z/a-z/' *.JPG

That’s it for now!

XKCD Widget for RPi

Get your daily dose of XKCD with this python based random comic widget. Designed to run on Raspberry Pi Official 7″ LCD (800×480) but easily adapted to other screens and other picture sources if you wish.

Get the latest Python file from my GitHub:

Open up the terminal, install python packages and create a directory to keep things neat and put the github file into it:

sudo apt-get install python-bs4
sudo apt-get install python-imaging python-pil.imagetk
cd /home/pi/
mkdir code
#Copy the file into this directory/folder from GitHub.

Test the file:

python /code/

Great, all should work, lets create a Desktop shortcut:

cd /home/pi/Desktop/
nano xkcd.desktop

To make the shortcut work paste in the below code to this file and Ctrl + x to exit.

[Desktop Entry]
Name=XKCD Widget
Comment=For Official RPI LCD.
Exec=python /home/pi/Desktop/code/

Give the shortcut executable permissions:

chmod +x xkcd.desktop 

Double click on the desktop icon and all should work. Currently do not have a logo for the icon but this is on the to do list.

RPi LCD Brightness Widget

I was surprised there was no way to adjust the backlight brightness out of the box for the official 7″ LCD (except for the command line obviously) so I made this widget to accomplish it.

Files available at my github:

First of all LCD brightness can be adjusted in the terminal using:

sudo sh -c 'echo "100" > /sys/class/backlight/rpi_backlight/brightness'

The range is 0-255 but nothing seems to happen after 200, if even get a little dimmer so my widget scales from 0-200. Put the .py & .png files in /home/pi/code directory. Put the .desktop file on your desktop. Make the files executable by:

cd /
chmod +x /home/pi/code/
chmod +x /home/pi/Desktop/brightness.desktop

The Desktop shortcut will work now but there was an annoying warning every time opening the file asking if wanted to execute the file in terminal etc., I got rid of it by opening the file manager, Go to Edit/Preferences/General, Check box for “Don’t ask options on launch executable file”

Now we have a shortcut that opens up the app to adjust the brightness but I wanted a taskbar shortcut:

Copy the .desktop file into the below two locations: (thanks to for guidance on this)

cp /home/pi/Desktop/brightness.desktop /usr/share/raspi-ui-overides/applications/
cp /home/pi/Desktop/brightness.desktop /usr/share/applications/

Open /home/pi/.config/lxpanel/LXDE-pi/panels/panel and add the below code:

nano /home/pi/.config/lxpanel/LXDE-pi/panels/panel
    Button {

Restart GUI without a full reboot:

 lxpanelctl restart && openbox --restart

All done, the shortcut should now be on the taskbar. Delete the desktop shortcut if not needed.

One improvement left to do would be to have the widget automatically set maybe 50% brightness in the event it was accidentally set to 0 previously.

ESP8266 logging to InfluxDB

The ESP8266 is a $5 IOT device with huge capabilities. In this post we will log data to a remote Influx database running on a RaspberryPi.

I am programming the ESP8266 in the Arduino IDE, the ESP8266 library is required, you can find it here. I have a test code file (of copy from below) that you can upload after entering your InfluxDB I.P. Address, SSID & Password and it will start logging data immediately.

#include <ESP8266WiFi.h>
#include <ESP8266WiFiMulti.h>
#include <InfluxDb.h>

#define INFLUXDB_HOST ""   //Enter IP of device running Influx Database
#define WIFI_SSID "SSID"              //Enter SSID of your WIFI Access Point
#define WIFI_PASS "PASSWORD"          //Enter Password of your WIFI Access Point

ESP8266WiFiMulti WiFiMulti;
Influxdb influx(INFLUXDB_HOST);

void setup() {
  Serial.print("Connecting to WIFI");
  while ( != WL_CONNECTED) {
  Serial.println("WiFi connected");
  Serial.println("IP address: ");


  Serial.println("Setup Complete.");

int loopCount = 0;

void loop() {

  InfluxData row("data");
  row.addTag("Device", "ESP8266");
  row.addTag("Sensor", "Temp");
  row.addTag("Unit", "Celsius");
  row.addValue("LoopCount", loopCount);
  row.addValue("RandomValue", random(10, 40));


The Arduino Serial Terminal will display something like the below so you can if it is working. (My previous tutorial shows setting up InfluxDB, ensure you have the database “esp8266_test” created as we are going to write to that.)

 --> writing to esp8266_test:
data,Device=ESP8266,Sensor=Temp,Unit=Celsius LoopCount=256.00,RandomValue=37.00
 <-- Response: 204 ""
 --> writing to esp8266_test:
data,Device=ESP8266,Sensor=Temp,Unit=Celsius LoopCount=257.00,RandomValue=20.00
 <-- Response: 204

On the Influx Database we can look at the data by:

USE esp8266_test
select * from data limit 50

Below you can see the export from my database (I have shortened the time field for neatness). You can see I reset the ESP8266 a couple of times due to the LoopCount value.

time        Device  LoopCount RandomValue Sensor Unit    
----        ------  --------- ----------- ------ ----   
52808175073 ESP8266 1         38          Temp   Celsius                              
63108846141 ESP8266 2         35          Temp   Celsius                              
69802517277 ESP8266 1         13          Temp   Celsius                              
79892112240 ESP8266 2         12          Temp   Celsius                              
89961602267 ESP8266 3         14          Temp   Celsius                              
99998928411 ESP8266 4         22          Temp   Celsius                              
10053683452 ESP8266 5         10          Temp   Celsius                              
20120378415 ESP8266 6         28          Temp   Celsius                              
30175745403 ESP8266 7         14          Temp   Celsius                              
40732248123 ESP8266 8         38          Temp   Celsius                              
51232948067 ESP8266 9         15          Temp   Celsius                              
61322347831 ESP8266 10        13          Temp   Celsius                              
71424432515 ESP8266 11        19          Temp   Celsius                              
84740185749 ESP8266 1         18          Temp   Celsius                              
94790343615 ESP8266 2         21          Temp   Celsius                              
04839215465 ESP8266 3         13          Temp   Celsius                              
31864448941 ESP8266 1         32          Temp   Celsius                              
41956355523 ESP8266 2         36          Temp   Celsius                              
52018136222 ESP8266 3         30          Temp   Celsius                              
62083037888 ESP8266 4         22          Temp   Celsius       

That’s it!

Backup InfluxDB

It makes sense to backup the InfluxDB periodically so we don’t loose all our data.

We can do this in the terminal by:

influxd backup -portable /home/pi/influx_backup/

Make it run every night at 2am by opening crontab and adding the below code:

crontab -e
0 2 * * * influxd backup -portable /home/pi/influx_backup/

Now it would make sense for the above location to be a USB drive etc. as if our main drive fails we would loose the backup along with the original data. We can do this by updating the crontab -e to:

0 2 * * * influxd backup -portable /media/YOUR_USB_DRIVE_NAME

I had to instal the below package to allow the RaspberryPi write to the USB drive:

sudo apt-get install ntfs-3g

Another improvement would be to put all this in a script and push to another machine maybe over FTP but this is as far as I got right now and works well.

We can also see how much data is on the USB Drive by the below, maybe we will log this to Influx in future to keep an eye on backup sizes.

du -sh /media/YOUR_USB_DRIVE_NAME

That’s it!

RPi Status Log to InfluxDB

In the last post we setup InfluxDB, now we are going to start storing system parameters every minute. It will work out of the box for Raspberry Pi and probably for some other Linux distros.

We are going to log the system uptime, the CPU & GPU Temperatures, the current CPU usage as well as the average CPU usage since boot.

The code is available here or copy from the end of this post. Put the code in a file called, don’t forget to update your IP address in the code and make it executable by:

chmod +x

We are going to log to database rpi_01, if you don’t have this created already complete the below:

create database rpi_01

Test our script run:


To confirm it works we can check the database:

use rpi_01
select * from system_status

and you should see something like: (type exit when you are done)

name: system_status
time   cpu_temp cpu_usage gpu_temp system   system_model   uptime
15329   39.5     14        40.1     RPI-01   ZeroW_V1.1   1386.68

Now we want the system status to be logged every minute, we do this by adding it to crontab:

crontab -e 

Add this line and save and close: (ensure path is correct to your file)

*/1 * * * * /home/pi/influx_scripts/

Check back after a while to ensure the logging is happening. In the next post we are going to show the status in graphical form using Grafana like the below: code:

# Gets SOC GPU Temperatures
gpu_temp_0=$(/opt/vc/bin/vcgencmd measure_temp | tr -cd '0-9.')

# Gets System Uptime
uptime=$(awk '{print $1}' /proc/uptime)

# Gets SOC CPU Temperatures
cpu_temp_0=$(cat /sys/class/thermal/thermal_zone0/temp)
cpu_temp_3=$(($cpu_temp_2 % $cpu_temp_1))

# Converts the total CPU Usage into %

  for i in {1..6}
  # Since the CPU fluctuates, it discards the first reading and averages the next 5.
  CPU=(`sed -n 's/^cpu\s//p' /proc/stat`) # Discards the cpu prefix
  IDLE=${CPU[3]} 			  # Just the idle CPU time.

  # Calculate the total CPU time.
  for VALUE in "${CPU[@]}"; do

  # Calculate the CPU usage since we last checked.

  # Remember the total and idle CPU times for the next check.

if [ $i -gt 1 ] # Ignores 1st reading as this is CPU average since boot
	let Average="$DIFF_USAGE+$Average"

  # Wait 1s before checking again.
  sleep 1

let Average="$Average/5"

curl -i -XPOST 'http://your.influxDB.ip.address:8086/write?db=rpi_01' --data-binary 'system_status,system=RPI-01,system_model=Insert_Model_Name cpu_usage='$Average',cpu_temp='$cpu_temp_4',gpu_temp='$gpu_temp_0',uptime='$uptime''

Credit to resources I used: